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ABSTRACT 


Successful  commercial  companies  understand  that  customers  are  the  real  experts 
with  regard  to  their  products  and  services.  Bringing  customer  experiences  right  into  the 
design  shop  allows  development  of  best-selling  commercial  products  and  services. 
Companies  such  as  L.L.  Bean,  Inc.  immerse  themselves  in  their  customer’s  experiences 
during  new  product  development.  They  travel  to  their  customer’s  location  and  listen  to 
them  face  to  face  to  get  the  best  possible  input  for  essential  product  requirements  and 
new  design  ideas. 

Currently,  most  military  C4I  systems  product  development  does  not  make 
effective  use  of  customer  input.  Systems  are  developed  and  fielded  in  accordance  with 
Department  of  Defense  regulations  that  provide  insufficient  mechanisms  for  users  to 
influence  product  requirements  and  design.  C4I  system  program  managers  need 
additional  tools  to  obtain  and  translate  user  needs  into  system  requirements  and  designs. 

Harvard  Business  School  has  developed  an  educational  program  aimed  at 
redesigning  product/service  development  based  on  the  L.L.  Bean  model.  This  thesis  will 
apply  the  tenets  of  that  program  to  submarine  C4I  systems  development  and  identify 
obstacles  to  and  lessons  learned  from  its  application  to  military  product  development. 
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I.  INTRODUCTION 


A.  BACKGROUND 

Customer- driven  product  development  is  a  key  component  of  successful  commercial 
business  practices  in  today’s  environment.  Organizations  such  as  L.L.  Bean,  Incorporated 
and  Ford  Motor  Company  integrate  and  ingrain  the  voice  of  the  customer  into  their  product 
development  culture.  Ford’s  Consumer  Relationship  Process  is  defined  by  “A  sustained 
relationship  with  the  consumer  through  the  development  of  consumer  insight,  which  is  used 
in  the  development  and  delivering  of  products  and  services”  (Renaud,  2000,  p.8).  This  type 
of  customer  focus  can  benefit  all  organizations,  both  private  and  government.  Ability  to 
meet  the  needs  cf  the  customer  can  be  the  difference  between  success  or  failure  in  the 
commercial  marketplace  and  in  government  systems  development.  Unfortunately,  many 
barriers  exist  to  successfully  integrating  the  voice  of  the  customer  into  military  systems 
development.  The  Government  Accounting  Office  (GAO)  has  issued  a  series  of  reports  on 
the  success  that  leading  commercial  firms  have  had  in  significantly  reducing  the  time  and 
money  needed  to  develop  new  and  more  sophisticated  products  -  the  same  results  that  the 
Department  of  Defense  (DoD)  desires.  Significant  cost  and  schedule  increases,  as  well  as 
significant  user  dissatisfaction,  can  be  traced  to  an  insufficient  definition  of  customer 
requirements  prior  to  program  management  decisions. 

The  basic  process  for  formulating  product  requirements  is  the  same  for  commercial 
firms  and  DoD.  Understanding  the  customer’s  expectations  is  the  first  step  for  both.  These 
expectations  must  then  be  translated  into  product  requirements,  including  the  functions. 
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characteristics,  reliability  and  maintainability  it  must  possess.  It  is  not  unusual  that  the  first 
understanding  of  customer  expectations  exceeds  the  capacity  of  the  product  developer  given 
the  current  state  of  technology  as  well  as  limited  resources  and  time  available.  Table  1 
highlights  the  contrasts  between  the  interests  of  the  customer  and  product  developer. 


Customer  wants _ 

Performance:  what  the  product  should  do. 
For  example,  what  an  aircraft’s  speed, 
range,  fuel  economy,  reliability  and  other 
features  should  be. 

Timing:  when  the  customer  wants  the 

product. 


Pricing:  what  the  product  will  cost.  The 
customer  must  be  able  to  afford  the 
product. 


Product  Developer’s  resources _ 

Technology:  the  technology  that  is  needed 
to  make  the  product  function  to  a  level 
necessary  to  meet  the  customer’s  wants. 

Schedule:  the  amount  of  time  that  is 

needed  to  develop,  design,  test  and 
manufacture  the  product. 

Investment  funds:  the  capital  that  is 

needed  to  pay  for  development,  test,  and 
manufacture  of  the  product. 

Expertise:  the  capabilities  of  the  product 
developer  in  terms  of  engineering 
expertise,  manufacturing  capabilities  and 
production. 


Table  1:  Customer  and  Product  Developer  Interests  in  a  Product’s  Developments 
(From:  GAO  Report  01-288,  March  2001) 

All  product  developments  attempt  to  match  customer  expectations  with  available 
resources  in  order  to  define  the  product.  The  matching  process  leads  to  close  scrutiny  of  the 
customer  expectations  and  developer’s  resources,  resulting  in  a  set  of  product  requirements 
that  lepresent  an  agreement  between  the  customer  and  developer  on  the  product’s  planned 
performance,  cost  and  schedule.  This  basic  requirements  process  is  depicted  in  figure  1 
below. 
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Figure  1:  The  Requirements  Process  (From:  GAO  Report  01-288,  March  2001) 

Translation  of  customer  expectations  into  a  set  of  specific  product  requirements 
includes  information  gathering,  analysis  and  negotiation.  In  commercial  product 
developments,  these  negotiations  occur  directly  between  the  customer  and  product  developer 
before  resources  are  committed  to  the  product  development.  During  this  process,  the  general 
customer  expectations  are  reduced  to  a  set  of  performance  requirements  that  are  achievable 
within  the  developer’s  available  resources  and  still  meet  the  customer’s  needs.  Two-way 
communications  occur  between  the  customer  and  product  developer  during  this  process.  For 
example,  during  new  commercial  aircraft  development,  the  customer  may  desire  a  certain 
speed  in  order  to  enhance  passenger  revenue,  but  the  developer  determines  that  the  time  and 
resources  available  are  insufficient  to  achieve  that  speed.  Negotiation  then  ensues  until  both 
parties  agree  upon  a  speed  requirement  that  satisfies  the  customer  need  within  the  available 
development  resources. 

DoD  employs  a  more  complex  requirements  process  involving  communication 
between  at  least  four  players.  Separating  the  customer  (an  organization  in  the  fighting 
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forces)  and  the  product  developer  (a  defense  contractor),  is  the  user  representative 
(organization  lepresenting  the  customer  in  product  requirements  negotiations)  and  the  DoD 
Program  Manager  (essentially  representing  the  product  developer  in  requirements 
negotiations)  (GAO  Report  01-288,  pp  18-19).  Figure  2  depicts  the  interplay  between  the 
major  players  in  the  commercial  and  DoD  requirements  processes. 


Commercial  example:  requirement  for  a  new  airliner 


Customei 

Product  Developer 

Airline  company 

- ► 

Aircraft  manufacturing  firm  responsible 

desiring  a  new 

for  developing  both  the  product 

passenger  aircraft. 

requirements  and  the  product  itself. 

DoD  example:  requirement  for  a  new  fighter  aircraft 


Figure  2:  Commercial  and  DoD  Organizations  Involved  in  Requirements 
Setting  Processes  (From:  GAO  Report  01-288,  March  2001) 

Both  commercial  and  defense  organizations  are  concerned  with  the  cost,  schedule  and 
performance  aspects  of  a  product’s  development,  and  these  concerns  are  reflected  in  the  final 
set  of  specific  product  requirements.  In  the  defense  process,  however,  the  ultimate  product 
developer  usually  has  little  input  into  the  requirements  development  process.  Formal  product 
requirements,  in  the  form  of  an  Operational  Requirements  Document,  are  provided  to  the 
prime  contractor  at  the  start  of  the  actual  product  development  cycle. 
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B.  PURPOSE 


The  purpose  of  this  thesis  is  to  evaluate  the  applicability  of  a  specific  commercial 
product  development  process  used  by  L.L.  Bean,  Incorporated  to  define  customer 
requirements  to  military  C4I  systems  development.  The  most  effective  means  of 
accomplishing  this  is  by  applying  the  tenets  of  that  process  to  the  development  of  specific 
military  C4I  systems.  Submarine  Exterior  Communications  Systems  have  been  selected  as 
the  candidate  systems  to  evaluate  this  approach  as  a  result  of  the  author’s  position  within  the 
Submarine  Communications  Program  Office  and  extensive  background  in  submarine 
operations  and  communications.  This  research  includes  interviews  with  submarine  officers 
and  enlisted  communications  personnel  currently  serving  aboard  fleet  units.  The  specific 
interview  questions  are  developed  using  the  L.L.  Bean  model. 

The  results  of  this  analysis  will  be  used  to  enhance  the  development  of  the  submarine 
C4I  systems  investigated  in  addition  to  assessing  the  applicability  of  the  process  to  military 
C4I  system  development.  Use  of  the  L.L.  Bean  product  development  process  should  allow 
for  enhanced  communication  of  customer  expectations  directly  to  program  office  personnel, 
thus  streamlining  the  requirements  negotiation  process  depicted  in  Ligure  2  above. 

C.  RESEARCH  QUESTIONS 

The  following  research  questions  fall  into  two  categories.  The  first  category  of 
questions  will  be  answered  primarily  through  review  of  current  literature.  The  second 
category  will  be  answered  by  conducting  interviews  with  submarine  C4I  system  users  using 
the  L.L.  Bean  method  and  analyzing  the  results. 
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1.  Research  Questions  Answered  by  Literature  Research 

Before  applying  the  L.L.  Bean  model  to  submarine  C4I  system  development,  an 
investigation  of  current  best  practices  in  commercial  customer- driven  product  development 
and  military  adaptation  of  these  commercial  best  practices  is  required.  Potential  benefits  and 
drawbacks  of  customer- driven  military  C4I  systems  product  development  should  also  be 
identified  and  researched.  The  following  questions  address  these  areas: 

1.  What  are  the  current  best  practices  in  customer- driven  commercial  product 
development? 

2.  How  have  commercial  best  practices  in  customer-driven  product  development 
been  adapted  for  military  use? 

3.  What  are  the  key  elements  to  customer-driven  product  development  (commercial 
or  military)? 

4.  What  are  the  benefits  of  customer- driven  military  C4I  systems  product 
development? 

5.  What  are  the  drawbacks  of  customer- driven  military  C4I  systems  product 
development? 

6.  What  are  the  special  considerations  required  for  customer- driven  military  C4I 
systems  product  development? 

2.  Research  Questions  Answered  by  Interview  Results 

The  crux  of  this  thesis  is  based  on  applying  the  L.L.  Bean  product  development 
interview  technique  to  submarine  C4I  systems  development.  The  following  questions  were 
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used  in  adapting  that  technique  to  gather  submarine  officer  and  enlisted  communications 
personnel  expectations  with  regard  to  submarine  C4I  systems. 

1 .  What  type  of  interview  questions  allows  users  to  convey  their  experiences  and 
ideas  regarding  military  C4I  systems  development  and  use? 

2.  What  is  the  best  way  to  conduct  interviews  with  users  to  enhance  military  C4I 
systems  product  development? 

3.  What  is  the  best  way  to  analyze  data  collected  during  user  interviews? 

4.  How  can  user  statements  during  interviews  be  translated  into  military  C4I  system 
product  requirements? 

5.  How  can  validated  product  requirements  be  translated  into  product  design  ideas? 

6.  How  can  product  design  ideas  be  incorporated  into  military  C4I  systems  product 
development? 

D.  BENEFITS  OF  STUDY 

The  benefits  of  this  study  are  twofold.  First  and  foremost,  the  applicability  of  the 
L.L.  Bean  interview  techniques  for  capturing  the  voice  of  the  customer  will  be  evaluated  in  a 
military  systems  development  environment.  If  successful,  this  technique  could  provide 
defense  program  managers  with  a  powerful  tool  to  help  define  customer  expectations  and 
negotiate  with  users  to  develop  product  requirements  that  satisfy  user  needs  within  the 
constraints  of  the  product  development  resources  and  timeframe  available  to  the  program 
manager. 

Secondly,  the  Submarine  Communications  Program  Manager  will  benefit  by 
receiving  direct  user  input  regarding  the  performance  and  suitability  of  submarine  C4I 
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systems  onboard  fleet  units  today.  This  information  should  enhance  the  program  manager’s 
decision-making  ability  regarding  planned  product  development  efforts,  ultimately  resulting 
in  improved  C4I  products  and  systems  and  enhancing  submarine  force  connectivity  in  the 
future. 

E.  SCOPE  AND  METHODOLOGY 

1.  Scope 

The  scope  of  this  thesis  is  divided  into  five  phases.  The  first  phase  involves  a  review 
of  commercial  product  development  best  practices  and  their  application  to  military  systems. 
This  phase  is  important  to  provide  an  understanding  of  successful  product  development 
practices  currently  in  use.  It  will  provide  a  baseline  assessment  for  how  well  customer 
desires  are  captured  and  included  in  both  the  commercial  and  military  product  development 
process. 

The  second  phase  consists  of  a  review  of  the  Harvard  Business  School’s  Redesigning 
Product/Service  Development  program  based  on  the  L.L.  Bean,  Incorporated  model.  This 
phase  is  necessary  to  understand  the  L.L.  Bean  method  of  incorporating  the  vrice  of  the 
customer  into  product  development  and  to  begin  adapting  that  method  for  military  C4I 
systems  development. 

The  third  phase  consists  of  application  of  the  tenets  of  the  Harvard  Business  School’s 
Redesigning  Product/Service  Development  program  to  submarine  C4I  systems  product 
development.  Gathering  data  on  customer  expectations  from  submarine  operators  through 
targeted  interview  questions  developed  during  phase  two  is  the  focus  of  this  phase. 
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The  fourth  phase  entails  an  in-depth  analysis  of  submarine  customer  interviews 
conducted  in  accordance  with  the  L.L.  Bean  model.  The  voice  of  the  submarine  customer 
can  thereby  be  determined  and  used  to  enhance  submarine  C4I  systems  development.  An 
assessment  of  the  applicability  of  the  L.L.  Bean  method  to  military  C4I  systems  development 
will  also  be  performed. 

The  fifth  and  final  phase  consists  of  translation  of  customer- expressed  needs  to 
submarine  C4I  systems  product  requirements  development  and  design.  This  phase  will  allow 
program  mangers  developing  submarine  C4I  systems  to  make  decisions  and  trade-offs 
regarding  cost,  schedule  and  performance  of  developments  under  their  cognizance. 

The  end  result  of  this  research  is  to  test  the  L.L.  Bean  product  development  model  in 
a  military  C4I  systems  environment.  It  will  conclude  with  recommendations  for  future 
applications  of  the  model. 

2.  Methodology 

The  methodology  used  in  this  thesis  research  consists  of  the  following  steps: 

1.  Conduct  a  literature  review  of  books,  magazine  articles,  CD-ROM  systems,  and 
other  library  information  resources  on  customer-driven  product  development  best 
practices. 

2.  Conduct  a  thorough  review  of  previous  research. 

3.  Conduct  a  review  of  the  Harvard  Business  School’s  Redesigning  Product/Service 
Development  program  based  on  the  L.L.  Bean  model. 

4.  Examine  the  current  military  C4I  systems  product  development  model. 
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5.  Develop  submarine  C4I  system  customer  interview  questions  using  the  Harvard 
Business  School  program  tenets. 

6.  Conduct  submarine  C4I  system  customer  interviews  in  accordance  with  the  L.L. 
Bean  model. 

7.  Thoroughly  analyze  submarine  C4I  system  customer  inputs  and  translate  them 
into  system  requirements  and  design  ideas. 

8.  Evaluate  efficacy  of  the  Harvard  Business  School  program  as  it  relates  to  military 
C4I  systems  development. 

F.  ORGANIZATION  OF  STUDY 

This  study  consists  of  six  chapters,  which  describe  current  best  practices  in 
commercial  product  development  and  their  application  to  military  systems,  the  Harvard 
Business  School’s  Redesigning  Product/Service  Development  program  based  on  the  L.L. 
Bean  model,  and  an  application  of  it  to  submarine  C4I  systems  development.  Chapter  I 
provides  a  brief  introduction  and  summary  of  this  thesis,  including  an  assessment  of  the  need 
for  improved  methods  of  collecting  customer  desires  and  expectations  in  military  systems 
development.  Chapter  II  consists  of  a  literature  review  on  best  practices  in  commercial 
product  development  and  their  application  to  military  systems.  The  L.L.  Bean  Product 
Development  model  is  also  described  in  detail  here. 

In  Chapter  III,  the  research  methodology  is  described.  This  chapter  explains  the  steps 
required  in  applying  the  L.L  Bean  model  to  submarine  C4I  system  product  development  and 
the  special  considerations  required.  Chapter  IV  provides  a  detailed  analysis  of  the  results  of 
submarine  customer  interviews.  Chapter  V  provides  an  evaluation  of  the  Harvard  Business 
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School’s  Redesigning  Product/Service  Development  program  application  to  submarine  C4I 
systems  product  development.  Finally,  Chapter  VI  summarizes  this  research,  provides 
lessons  learned  and  recommends  further  research  areas. 
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II.  LITERATURE  REVIEW 


A.  INTRODUCTION 

In  product  development,  satisfying  customer  needs  is  recognized  as  a  key  competitive 
success  factor  in  today’s  commercial  marketplace.  Product  development  teams  consisting  of 
engineers  and  designers  must  be  able  to  extract  a  high  degree  of  detail  regarding  customer 
expectations  in  order  to  be  successful.  A  laser- like  focus  on  the  customer  is  necessary  to 
determine  the  best  possible  answers  to  the  following  product  development  relevant  questions: 
(1)  What  exactly  do  customers  value?  (2)  From  these  values,  what  are  the  product 
requirements  that  will  achieve  the  highest  possible  level  of  customer  satisfaction?  (Renaud, 
2000).  The  voice  of  the  customer  can  provide  a  new  product  idea  or  suggest  innovations  for 
an  existing  product.  Effectively  engaging  the  customer  voice  enables  a  company  or 
government  program  office  to  better  understand  how  customers  use  their  products.  A 
product’s  success  (or  failure)  can  be  a  direct  result  of  the  company /program  office’s  ability  to 
understand  and  satisfy  the  voice  of  the  customer. 

To  date,  the  degree  of  improvement  in  new  product  development  methods  has  been 
higher  in  the  commercial  sector  than  the  defense  sector.  The  commercial  world  has 
integrated  the  customer  voice  throughout  the  organization  as  the  primary  means  to  improve 
product  development.  In  DoD,  institutional  barriers  can  hinder  widespread  integration  of  the 
voice  of  the  customer  in  product  development.  Government  product  developers  tend  to  focus 
on  development  process  and  methods,  providing  less  interaction  with  users  during  product 
development. 
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This  chapter  examines  several  current  best  practices  in  commercial  product 
development  and  their  application  to  military  systems.  It  discusses  how  successful 
companies  integrate  the  voice  of  the  customer  into  their  product  development  strategies,  and 
how  government  program  offices  could  benefit  from  similar  strategies.  The  chapter  then 
focuses  on  the  Harvard  Business  School’s  Redesigning  Product/Service  Development 
program,  describing  how  L.L.  Bean,  Incorporated  integrates  the  voice  of  the  customer  into 
their  product  development  process  and  how  to  apply  the  same  principles  to  military 
product/service  development  efforts. 

B.  COMMERCIAL  BEST  PRACTICES 

Many  different  techniques  are  employed  by  commercial  organizations  to  obtain  and 
use  the  voice  of  the  customer  during  the  product  development  process.  Often,  the  method 
used  to  gain  insight  into  the  customer  voice  depends  on  the  company’s  specific  objective  and 
type  of  information  desired.  Companies  seeking  meaningful  feedback  from  a  relatively  large 
group  of  customers  at  once  may  employ  the  focus  group  method.  Survey  methods  can 
provide  insight  on  how  customers  rate  specific  product  features.  Worldwide  Web-based 
methods,  such  as  the  Sloan  School  of  Management’s  Information  Pump  have  also  been 
developed.  Finally,  one-on-one  interviews  can  be  used  to  provide  detailed  customer 
expectation  data  from  individual  users.  Each  of  these  methods  has  distinct  strengths  and 
weaknesses  as  discussed  below. 

I.  Focus  Groups 

Product  development  practices  using  focus  groups  bring  together  multiple  users  in  a 

single  setting  to  gain  insight  from  the  group  as  a  whole.  A  moderator  is  usually  employed  to 
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lead  group  discussions  directed  at  obtaining  customer  expectations  with  regard  to  product 
features  under  consideration  by  the  company’s  product  development  team.  Focus  group 
feedback  can  provide  a  good  indication  regarding  how  the  larger  customer  population  will 
likely  react,  allowing  the  company  to  evaluate  whether  or  not  they  should  proceed  with  a 
particular  idea.  A  significant  advantage  of  the  focus  group  technique  is  that  it  allows  for  a 
variety  of  perspectives  to  be  advanced  and  discussed  in  an  open  forum,  often  leading  to  frank 
and  constructive  product  development  inputs.  However,  the  focus  group  technique  can  suffer 
if  one  or  more  individuals  dominate  the  discussion  and  influence  the  overall  group  response. 
In  most  cases,  “the  focus  group  can  provide  helpful  insight  into  the  voice  of  the  customer” 
(Mastronardi,  2001,  p.  17). 

2.  Customer  Surveys 

Surveys  can  be  provided  to  either  a  target  set  of  customers  or  randomly  to  gain 
insight  regarding  how  customers  might  perceive  product  or  service  features.  Many  types  of 
surveys  and  methods  of  administering  them  are  currently  in  use.  Telephones,  the  Internet, 
mailings  and  face-to-face  surveys  at  the  point  of  sale  can  all  be  effective  in  capturing  the 
voice  of  the  customer.  Advantages  of  the  survey  method  include:  (1)  detailed  data  regarding 
customer  perceptions  of  a  product  or  service  can  be  obtained  and  (2)  statistically  relevant 
metrics  can  be  developed  and  displayed  conveniently  for  decision-makers.  The  primary 
disadvantages  of  the  survey  method  include  the  potential  for  a  high  numbers  of  non¬ 
responses  and  customer  misunderstanding  of  questions  leading  to  an  incorrect  assessment  of 
customer  product  desires.  Surveys  can  be  useful  in  providing  customer  voice  information  to 
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companies,  but  their  use  is  generally  limited  to  more  targeted  roles  than  other  methods 
(Mastronardi,  2001). 

3.  Information  Pump 

Drazen  Prelec  (Professor  of  Management  Science,  Sloan  School  of  Management, 
Massachusetts  Institute  of  Technology)  has  developed  a  method  called  the  Information  Pump 
focusing  on  providing  incentives  for  collecting  customer  information.  These  incentives  are 
integrated  into  a  web-based  information  system  for  data  collection  relevant  to  the  product 
design  process.  This  incentive -based  approach  has  the  advantage  of  allowing  for  consistent 
motivation  and  effort  by  the  customer,  and  the  incentives  clearly  communicate  the 
information  desired  from  customers.  By  allowing  customers  to  monitor  their  progress, 
improved  responses  can  be  obtained  over  time.  This  approach  also  identifies  the  most 
responsive  customers  who  can  then  be  used  for  later  product  development  efforts.  Two 
different  approaches  can  be  used  to  create  incentives  with  customers:  (1)  linking 
compensation  to  the  outcome  of  the  project  and  (2)  enabling  the  comparison  of  responses 
from  two  customers  simultaneously  via  the  Internet.  The  Information  Pump  can  provide  the 
voice  of  the  customer  via  the  Web  and  enable  customers  to  compare  and  contrast  desirable 
product  features  via  web-based  interaction  with  one  another.  This  method  also  has  the 
distinct  advantage  of  incorporating  a  scoring  method  for  qualitative  and  quantitative 
feedback  (Mastronardi,  2001). 

4.  One  -o  n-One  Interviews 

Direct  interviews  with  targeted  customers  provide  unfiltered  feedback  to  companies 

regarding  customer  expectations  that  can  be  extremely  useful  to  product  development  teams. 
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Questions  posed  to  the  interviewees  are  often  open-ended  to  allow  the  interviewee  to  take  the 
lead  in  discussions.  An  interview  guide,  developed  beforehand,  allows  the  interviewer  to 
keep  the  discussions  on  track  with  the  research  objectives.  The  voice  of  the  customer  may 
express  a  product  need,  an  area  where  a  specific  product  or  service  fell  short  of  expectations 
or  even  a  product  feature  that  created  customer  delight.  One-on-one  interviews  can  be 
invaluable  in  helping  companies  determine  how  their  products  are  actually  used  and  how 
they  fit  into  their  customers’  lifestyle.  The  success  of  one-on-one  interviews  in  revealing  the 
voice  of  the  customer  is  directly  dependent  on  the  structuring  of  interview  questions  to  meet 
the  desired  objectives  (Mastronardi,  2001).  The  Harvard  Business  School’s  Redesigning 
Product/Service  Development  method  relies  strongly  on  one-on-one  interviews  to  understand 
customer  expectations  and  integrate  the  voice  of  the  customer  into  the  product  development 
process. 


C.  GOVERNMENT  ADOPTION  OF  COMMERCIAL  BEST  PRACTICES 

For  several  years,  the  Government  Accounting  Office  (GAO)  has  been 
recommending  the  adoption  of  commercial  best  practices  for  DoD  product  development. 
Their  opinion  is  “the  best  practices  of  leading  commercial  firms  can  be  used  to  improve  the 
development  of  technology  and  weapon  systems  in  DoD”  (GAO  Report  99-116,  p.  1).  The 
best  practices  model  advocated  by  GAO  is  based  on  developing  knowledge  to  answer  the 
basic  question  of  how  a  capability  can  best  be  provided  to  the  customer.  Understanding  the 
voice  of  the  customer  is  a  crucial  first  step  in  developing  the  requisite  knowledge  base 
advocated  by  the  GAO.  DoD  regulations  also  recognize  the  importance  of  customer  input. 
“The  Defense  acquisition  and  requirements  communities  shall  maintain  continuous  and 
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effective  communications  with  each  other  and  with  the  operational  user.  The  objective  is  to 
gain  a  sound  understanding  of  the  users’  needs  and  to  work  with  them  to  achieve  a  proper 
balance  among  cost,  schedule,  and  performance  considerations.”  (DoD  Instruction  5000.1, 
Paragraph  4.2.2).  To  help  foster  the  required  “effective  communications”  with  operational 
users,  acquisition  program  offices  could  employ  the  Harvard  Business  School’s  Redesigning 
Product/Service  Development  method  being  evaluated  herein. 

GAO  has  also  identified  several  successful  DoD  initiatives  where  commercial  best 
practices  have  been  adopted  to  enhance  military  system  development.  Examples  include 
maturing  technologies  prior  to  incorporating  them  into  programs  of  record  (e.g.,  VIRGINIA 
(SSN  774)  Photonics  Mast,  the  Advanced  Amphibious  Assault  Vehicle  propulsion 
technology,  and  the  Air  Force’s  Integrated  High  Performance  Turbine  Engine  Technology 
Program),  emphasizing  cost  as  an  independent  variable  and  utilizing  integrated  product  teams 
to  manage  system  development  (GAO  Report  99-116).  Government  program  office  success 
in  adopting  some  commercial  best  practices  indicate  that  others,  such  as  those  used  to  obtain 
and  integrate  the  voice  of  the  customer  into  product  development,  can  be  effectively  adapted 
to  enhance  DoD  processes. 

D.  HARVARD  BUSINESS  SCHOOL’S  REDESIGNING  PRODUCT/SERVICE 

DEVELOPMENT  PROGRAM 

Harvard  Business  School  has  developed  a  program  titled  “Redesigning 
Product/Service  Development”  based  on  the  E.E.  Bean,  Incorporated  process  for  obtaining 
the  voice  of  the  customer.  This  program  provides  a  toolkit  to  guide  product  developers  in  its 
implementation  in  their  own  setting.  The  toolkit  contains  a  video  introducing  the  benefits 
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and  steps  of  customer-driven  product  or  service  development,  using  L.L.  Bean,  Inc.,  as  an 
example,  a  blueprint  detailing  how  to  apply  the  video  techniques  to  other  product  or  service 
development  challenges  and  worksheets  to  facilitate  organization  of  the  customer-driven 
product  development  work.  The  material  in  the  toolkit  is  based  on  the  writing  and  research 
of  Professor  David  A.  Garvin.  This  program  is  described  in  detail  in  the  paragraphs  below. 

1.  L.L.  Bean  Model 

The  key  points  of  the  Harvard  Business  School  customer- driven  product  development 
model  can  be  summarized  as  follows  (Garvin,  2000): 

a.  Customers  are  the  real  product  or  service  experts. 

b.  Developing  the  best  products  or  services  requires  understanding  and  bringing 
customer  experiences  right  into  the  design  shop. 

c.  Don’t  isolate  product  developers  from  customers  with  outside  market  researchers 
or  focus  group  one-way  mirrors.  Immerse  developers  in  customer  experiences. 

d.  Go  right  to  the  customer’s  turf. 

e.  Listen  to  customers  face  to  face. 

f  Watch  customers  actually  using  products  or  services. 

g.  Translate  what  you  see  and  hear  into  the  customer’s  essential  product  or  service 
requirements 

h.  Develop  product  or  service  designs  that  precisely  meet  those  essential 
requirements 

i.  Highlight  this  customer- driven  approach  in  your  marketing. 
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The  video  provided  in  the  Harvard  Business  School  program  toolkit  depicts  actual 
L.L.  Bean  product  developers  applying  these  key  points  to  the  development  of  new  products. 
The  blueprints  and  worksheets  provide  a  framework  for  adapting  these  tenets  and  methods  to 
other  products  and  services. 

2.  Criteria  for  Field  Testers 

The  early  stages  of  customer- driven  product  or  service  development  rely  on  a  very 
small  group  of  customers  (approximately  16-20).  If  properly  selected,  this  group  can  be  used 
again  and  again  to  help  develop  new  products  or  services  and  improve  existing  ones.  Thus, 
field  tester  selection  is  a  critical  part  of  the  process.  The  goal  is  to  form  a  group  of  field 
testers  who  are:  (1)  experienced  users  of  the  target  product/services,  (2)  demographically 
diverse,  and  (3)  articulate,  honest  communicators.  The  Harvard  Business  School  program 
begins  with  the  definition  of  field  tester  selection  criteria,  and  then  applies  those  criteria  to 
select  field  testers  from  the  customer  base.  This  process  proceeds  in  the  following  steps 
(Garvin,  2000): 

a.  Definition  of  users  versus  purchasers  -  although  it  may  be  useful  to  talk  with 
purchasers  of  products  and  services,  it  is  essential  to  listen  to  and  watch  customers  who 
actually  use  the  products  and  services.  For  example,  parents  purchase  toys,  but  children  are 
the  actual  users  of  toys. 

b.  Definition  of  experienced  users  -  only  about  16-20  customers  will  become  field 
testers,  so  it’s  important  to  choose  them  from  the  most  experienced  product/service  users. 
For  example,  experienced  users  may  be  defined  as  those  customers  who  use  X  quantity  of 
products  or  services  within  Y  period  of  time. 
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c.  Definition  of  demographic  categories  from  which  diverse  representation  is  needed 
among  experienced  users  -  ensuring  diversity  among  experienced  users  will  provide  a 
broader  range  of  perspectives  regarding  the  products  or  services.  For  example,  demographic 
categories  can  be  defined  by  occupation,  gender,  age,  etc. 

d.  Definition  of  common  characteristics  for  experienced,  demographically  diverse 
users  -  the  most  valuable,  experienced,  demographically  diverse  users  should  have  certain 
characteristics  in  common.  For  example,  L.L.  Bean  ensures  its  field  testers  are  articulate 
communicators,  analytical  in  comparison  of  their  products,  honest  and  unbiased,  and  timely 
in  providing  feedback. 

3.  Field  Tester  Interviews 

The  field  tester  interview  process  described  in  the  Harvard  Business  School’s 
Redesigning  Product/Service  Development  program  is  based  on  open-ended  interviewing 
techniques  that  have  roots  in  anthropology  (see  Schwartzman,  pp.  58-60,  66).  Two  key  areas 
of  focus  for  the  field  tester  interviews  include  the  interview  questions  and  the  interview  pairs. 

The  interview  questions  should  enable  the  field  testers  to  convey  the  depth  of  their 
experiences  with  the  product/service  by  vividly  describing  what  happens  and  how  they  feel 
when  using  them,  explaining  what  problems  they’ve  had  in  using  them,  and  expressing  their 
excitement  regarding  them.  The  questions  must  be  broad  and  open-ended  to  promote  in- 
depth  answers,  be  non- leading  so  as  not  to  suggest  any  particular  answer,  explore 
comparison’s  with  other  products/services  and  take  no  more  than  an  hour  of  the  field  tester’s 
time.  The  final  list  of  question  should  be  narrowed  down  to  no  more  than  five  or  six.  L.L. 
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Bean  uses  the  following  types  of  questions  when  interviewing  hunters  who  use  their  boots 
(Garvin,  2000,  p.  29): 

-  “Tell  me  your  best  hunting  story.” 

-  “Describe  a  situation  in  which  your  hunting  footwear  let  you  down.” 

-  “If  you  could  design  your  own  custom  hunting  boots,  what  would  they  be  like?” 

-  “What  haven’t  I  asked  you  about  your  footwear  that  you’d  like  to  discuss?” 

The  Harvard  Business  School  program  also  stresses  the  need  to  conduct  the 
interviews  using  a  pair  of  interviewers  with  distinct  roles  -  an  interviewer  and  a  recorder. 
The  interviewer  encourages  field  testers  to  describe  their  experiences  and  feelings  to  say 
what  is  on  their  minds,  listens  attentively,  never  interrupts  and  redirects  the  interviewee  only 
when  absolutely  necessary.  For  example,  the  interviewer  might  ask  the  field  tester  to  say 
more  about  his  or  her  experiences  if  the  tester  had  jumped  to  product/service  solutions  prior 
to  providing  a  clear  picture  of  their  specific  experience.  The  recorder  remains  silent  and 
attentive,  writes  down  exactly  what  the  field  tester  says,  doesn’t  interpret,  filter  or  summarize 
the  field  tester’s  words,  and  asks  for  clarification  if  he  doesn’t  understand  the  tester’s 
comments.  The  method  also  allows  the  recorder  to  discreetly  operate  a  small  recording 
device  if  the  field  tester  is  comfortable  being  recorded,  but  it  notes  that  most  people  prefer 
not  to  be  recorded.  A  recorded  transcript  of  the  interview  can  be  useful,  but  it  is  not  essential 
if  the  written  transcript  is  carefully  maintained  (Garvin,  2000). 

4.  Interview  Data  Analysis 

Analysis  of  the  data  collected  during  field  tester  interviews  is  conducted  in  two  parts: 
(1)  post- interview  debriefings  and  (2)  summarizing  field  tester  quotations  and  descriptions. 
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Post- interview  debriefings  should  be  conducted  as  soon  as  possible  after  each  interview.  The 
debriefing  allows  the  interviewer  and  recorder  to  review  their  notes  and  fill  in  any  gaps.  As 
they’re  reviewing,  they  can  also  begin  to  capture  what  the  field  testers  think  and  feel  most 
deeply  about  their  products/services,  selecting  evocative  quotations  and  vivid  descriptions 
and  recording  them  on  sheets  of  paper  headed  with  each  interview  question.  For  example, 
L.L.  Bean  hunter  interviews  provided  the  following  types  of  evocative  quotations  and  vivid 
descriptions  (Garvin,  2000,  p.  33): 

-  “walking  through  wet,  swampy  areas  -  water  seeping  into  my  boots” 

-  “pulling  socks  way  up  because  the  boot  rubs  and  rubs  and  rubs  -  hair  rubbed  right 
off’ 

-  “putting  on  wet,  cold  boots  in  the  morning  -  just  miserable” 

The  next  step  is  to  summarize  and  translate  these  field  tester  quotations  and 
descriptions  using  the  following  steps  (Garvin,  2000): 

a.  Pool  quotations  and  descriptions  -  posting  each  interview  question  on  the  wall  and 
putting  each  quotation  and  description  under  the  appropriate  question  provides  one  method  to 
accomplish  this. 

b.  Prioritize  and  Reduce  pooled  quotations  and  descriptions  -  apply  your 
understanding  of  field  tester  priorities. 

c.  Group  prioritized  quotations  and  descriptions  -  remove  the  interview  questions 
from  the  wall  and  rearrange  the  quotations  and  descriptions  into  clusters  by  themes. 

d.  Summarize  grouped  quotations  and  descriptions  -  distill  the  essence  of  each  cluster 
of  field  tester  quotations  and  descriptions  into  a  brief  statement.  For  example,  the  first  and 
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third  quotation  listed  earlier  in  this  section  regarding  L.L.  Bean  hunting  boots  could  be 
grouped  under  the  summary  statement,  “Feet  wet  and  cold”. 

5.  Translation  of  Interview  Results  into  Product  Requirements  and  Design 
Ideas 

Each  quotation/description  summary  statement  developed  from  the  field  tester 
interview  analysis  is  then  translated  into  many  field  tester  requirements.  For  example,  the 
summary  statement,  “Feet  wet  and  cold”  could  be  translated  into  two  requirements  for  the 
hunter’s  footwear:  (1)  keep  the  feet  dry  and  (2)  keep  the  feet  warm.  All  of  the  product 
requirements  thus  generated  are  then  pooled,  prioritized  and  reduced,  grouped  into  prioritized 
requirements  and  then  further  prioritized  and  reduced.  The  final  set  of  prioritized  product 
requirements  should  not  contain  any  redundant  requirements  and  should  accurately  capture 
the  field  tester’s  priorities.  After  obtaining  a  set  of  valid  requirements,  each  is  translated  into 
many  design  ideas.  For  example,  the  requirement  to  keep  the  feet  warm  might  create  design 
ideas  for  flannel  sock  liners  or  insulated  boot  inserts.  All  of  the  design  ideas  thus  generated 
are  pooled,  prioritized  and  reduced,  grouped  into  prioritized  design  ideas  and  then  further 
prioritized  and  reduced.  The  final  set  of  grouped  design  ideas  should  not  contain  any 
redundant  ideas,  should  precisely  meet  the  valid  requirements  and  be  logistically  realistic. 
The  final  set  of  design  ideas  should  also  be  compared  to  the  field  tester  quotations  and 
descriptions  to  ensure  that  the  ideas  and  concerns  of  the  experienced,  demographically 
diverse  product/service  users  are  still  driving  the  development  (Garvin,  2000). 
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E.  SUBMARINE  C4I  SYSTEM  S  OF  INTEREST 

Three  submarine  C4I  systems  of  interest  were  selected  for  this  study  to  assess  the 
applicability  of  the  Harvard  Business  School’s  Redesigning  Product/Service  Development 
program  and  to  provide  useful  feedback  to  the  Space  and  Naval  Warfare  Systems  Command 
Submarine  Communications  Program  Office  regarding  planned  product  enhancements  for 
currently  fielded  systems  and  future  new  product  developments.  Each  system  and  the  reason 
for  its  selection  are  described  briefly  in  the  paragraphs  below. 

I.  Ultra  High  Frequency  (UHF)  Medium  Data  Rate  (MDR)  Capability 

Installation  of  UHF  MDR  capability  on  submarines  provides  a  32  kilobit  per  second 
(kbps)  UHF  asymmetric  satellite  communications  capability  to  support  Internet  Protocol  (IP) 
connectivity,  enabling  web  browsing  and  e-mail  SIPRNET  services  for  the  submarine  while 
underway.  UHF  MDR  is  installed  on  all  fast  attack  submarines  deploying  with  Battle 
Groups  to  provide  an  interim  medium  data  rate  IP  capability  prior  to  the  fielding  of 
Submarine  High  Data  Rate  antenna.  The  program  office  perceives  the  UHF  MDR  program 
as  an  acquisition  success  story  due  to  it’s  relatively  low  cost,  it’s  rapid  speed  to  deployed 
capability  and  it’s  importance  to  future  communications  concepts  cf  operations.  It  is  of 
interest  to  this  study  and  to  the  Submarine  Communications  Program  Office  to  determine 
how  well  the  UHF  MDR  capability  was  meeting  the  fleet  need  for  SIPRNET  connectivity 
and  obtain  the  voice  of  the  submarine  customer  perspective  regarding  needed  upgrades  for 
the  system. 
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2.  Mini-DAMA  UHF  Transceiver 


The  AN/USC-42(V)  Miniaturized  Demand  Assigned  Multiple  Access  (Mini-DAMA) 
Program  consists  of  Non- Developmental  Item/Commercial  Off-The-Shelf  (NDI/COTS) 
equipment  to  provide  a  communication  system  that  supports  the  exchange  of  secure  and  non- 
secure  Battle  Group  coordination  data,  tactical  data  and  voice.  This  system  is  of  interest  to 
this  study  because  of  the  multiple  design  and  logistics  problems  involved  with  its  fielding  on 
operational  submarines  and  the  desire  to  capture  the  voice  of  the  submarine  customer  during 
development  of  planned  field  changes/upgrades. 

3.  Submarine  Tactical  Data  Links 

Submarine  Tactical  Data  Links  currently  in  use  onboard  operational  submarines 
include  LINK  11,  LINK  16  and  Satellite  TADIL-J  (S-TADIL-J).  These  systems  allow  near 
real  time  transfer  of  tactical  track  data  (course,  speed,  identification,  etc.  of  contacts)  between 
submarines  and  other  Battle  Group  units.  These  systems  are  of  interest  to  this  study  to  help 
evaluate  which  of  these  systems  submarine  customers  actually  use  and  to  capture  the  voice  of 
the  submarine  customer  in  defining  product  requirements  for  systems  under  currently  under 
development  to  replace  these  older,  legacy  tactical  data  transfer  systems. 

F.  CHAPTER  SUMMARY 

This  chapter  reviewed  commercial  best  practices  used  to  integrate  the  voice  of  the 
customer  into  product  development,  along  with  successful  examples  of  DoD  adaptation  of 
other  commercial  best  practices.  The  Harvard  Business  School’s  Redesigning 
Product/Service  Development  program  based  on  the  L.L.  Bean,  Incorporated,  product 
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development  model  was  described  in  detail.  This  method  emphasizes  face-to-face  interviews 
with  experienced  product  users,  called  field  testers,  using  open-ended  questions  conducted  by 
two  person  interview  teams.  Interview  data  are  analyzed  to  provide  field  tester  quotations 
and  descriptive  phrases  that  are  further  developed  into  product/service  requirements  and 
design  ideas.  This  chapter  also  provided  a  brief  description  of  submarine  C4I  systems  of 
interest  that  will  serve  as  the  test  case  for  application  of  this  commercial  product 
development  process  to  military  C4I  systems  development.  The  remaining  chapters  of  this 
research  will  describe  the  application  of  the  Harvard  Business  School  process  to  the  conduct 
of  submarine  user  interviews  and  development  of  submarine  C4I  system  product 
requirements  and  design  ideas. 
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III.  RESEARCH  METHODOLOGY 


A.  INTRODUCTION 

The  purpose  of  this  research  is  to  apply  the  Harvard  Business  School’s  Redesigning 
Product/Service  Development  method  of  obtaining  the  voice  of  the  customer  to  submarine 
C4I  systems  development,  evaluating  the  applicability  of  this  commercial  product 
development  process  to  military  systems  development.  The  research  methodology  is  divided 
into  three  main  areas  as  defined  by  the  Harvard  Business  School  program.  The  first  area 
involves  planning  the  product  development  interviews  to  include  determining  the 
composition  of  the  interview  team  and  developing  the  questions  to  be  used  during  the 
interview.  Because  the  goal  is  to  assess  the  applicability  of  the  Harvard  Business  School 
program  to  military  systems  development,  the  tenets  of  that  program  are  followed  as  closely 
as  possible  in  planning  the  submarine  customer  interviews.  Careful  preparation  and 
development  of  the  interview  guide  will  help  keep  the  subsequent  interviews  focused  and 
effective. 

The  second  major  area  of  research  methodology  involves  selecting  field  testers  from 
the  submarine  community  to  hterview.  Again,  the  Harvard  Business  School  process  is 
followed  as  closely  as  possible  and  adapted  where  necessary  to  include  submarine -specific 
criteria.  Proper  definition  of  submarine  field  tester  criteria  is  critical  to  ensure  useful 
feedback  is  obtained  regarding  the  selected  submarine  C4I  systems  of  interest  and  to 
adequately  assess  the  applicability  of  this  commercial  product  development  process  to 
military  systems. 
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The  final  area  of  research  methodology  is  the  conduct  of  the  interviews  themselves. 
In  order  to  analyze  the  effectiveness  of  the  L.L.  Bean  product  development  method  in  this 
context,  feedback  must  be  obtained  directly  from  the  selected  submarine  field  testers.  The 
interview  questions  developed  in  accordance  with  the  Harvard  Business  School  program 
guidelines  are  used  to  guide  the  discussions  with  the  submarine  field  testers  and  their 
responses  are  recorded  exactly.  In  order  to  accurately  capture  the  voice  of  the  submarine 
customer,  the  interviews  are  conducted  on  their  home  turf  aboard  their  submarines.  The 
results  of  these  interviews  help  generate  conclusions  and  recommendations  regarding  the 
applicability  of  the  Harvard  Business  School  program  to  military  systems  development  and 
provide  useful  data  for  the  Submarine  Communications  Program  Office  efforts  to  improve 
existing  products  and  develop  new  ones. 

B.  PLANNING  PRODUCT  DEVELOPMENT  INTERVIEWS 

The  first  step  in  applying  the  Redesigning  Product/Service  Development 
methodology  is  to  plan  the  product  development  interviews.  Two  key  areas  of  initial 
interview  planning  are  the  composition  of  the  interview  team  and  the  development  of 
interview  questions  used  to  guide  the  discussions  with  field  testers.  Both  of  these  areas  are 
discussed  separately  below. 

I.  Interview  Team  Composition 

The  Harvard  Business  School’s  Redesigning  Product/Service  Development  program 
is  quite  specific  regarding  the  necessary  interview  team  composition.  Two  persons  make  up 
the  interview  team  -  one  takes  the  role  of  interviewer  and  the  other  acts  as  recorder.  The 


interviewer  facilitates  the  interview  using  open-ended  questions  as  a  guideline  and 
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encouraging  field  testers  to  vividly  describe  their  experiences  and  feelings  regarding  the 
product  being  discussed.  The  recorder  remains  silent  during  the  interview  except  to  ask  for 
clarification  when  required  and  writes  down  exactly  what  the  field  tester  says. 

For  the  submarine  customer  product  development  interviews,  the  author  acted  as 
interviewer,  applying  his  fifteen  years  of  submarine  operating  experience  and 
communications  systems  expertise  to  enhance  the  conduct  of  the  interviews.  Selected 
submarine  field  testers  appreciated  the  interviewer’s  detailed  understanding  of  their  operating 
environment  and  assigned  missions.  Due  to  the  technical  nature  of  the  interview  discussions, 
it  was  important  that  the  interviewer  be  a  subject  matter  expert  in  the  area  of  submarine 
operations  and  communications.  A  group  discussion  facilitator  (Mr.  Gary  Rossi)  from  Booz, 
Allen  and  Hamilton,  Inc.  was  selected  to  act  as  the  recorder  for  the  submarine  customer 
product  development  interviews.  Mr.  Rossi  also  had  previous  experience  as  a  Naval  Officer 
prior  to  his  employment  at  Booz,  Allen  and  Hamilton.  The  interview  team  worked  closely 
together  to  learn  and  apply  the  tenets  of  the  L.L.  Bean  product  development  process  to  plan 
and  conduct  the  interviews. 

2.  Interview  Questions 

The  L.L.  Bean  product  development  interview  process  as  described  in  the  Harvard 
Business  School’s  Redesigning  Product/Service  Development  program  is  also  very  specific 
with  regard  to  interview  question  generation.  In  order  to  promote  detailed,  in-depth 
discussions  with  the  submarine  field  testers,  the  questions  need  to  be  broad  and  open-ended. 
They  must  not  lead  the  interviewees  to  any  particular  answer  and  should  take  no  more  than 
one  hour  of  the  submarine  field  testers’  time.  The  goal  of  the  interview  questions  is  to  evoke 
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vivid  descriptions  from  the  field  testers  regarding  what  happens  and  how  they  feel  when 
using  the  submarine  C4I  systems  of  interest  described  earlier  and  explaining  what  problems 
they  have  when  using  them. 

Using  these  guidelines,  the  submarine  customer  product  development  interview  team 
derived  the  following  set  of  questions  that  were  used  to  guide  the  course  of  all  subsequent 
interviews  with  submarine  field  testers.  These  questions  were  designed  to  gather  information 
regarding  the  operability  and  maintainability  of  the  C4I  systems  of  interest  installed  prior  to 
the  most  recently  completed  period  of  deployed  operations  as  well  as  the  submarine  customer 
voice  regarding  the  installation  process. 

a.  Describe  the  most  demanding  communications  period  that  occurred  during  the 
deployment  and  how  well  or  how  poorly  the  new  C4I  system  installations  supported  the  ship 
during  that  period? 

b.  Describe  the  most  difficult  maintenance  evolution  involving  the  new  C4I  systems 
that  occurred  during  the  deployment  and  how  well  or  how  poorly  the  technical 
documentation  supported  your  maintenance  efforts. 

c.  Describe  the  training  provided  during  the  installation  of  the  new  C4I  systems  and 
how  well  or  how  poorly  that  training  prepared  you  for  operating  and  maintaining  the 
equipment  during  your  deployment. 

d.  Describe  the  impact  of  the  new  C4I  system  installation  on  the  ship’s 
routine/schedule . 

e.  Are  there  any  other  issues  or  problems  with  the  new  C4I  systems?  Were  any 
Casualty  Reports  (CASREPS)  involving  the  new  C4I  systems  required  during  the 


deployment? 
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Appendix  A  provides  an  example  of  the  interview  data  sheets  developed  as  part  of 
this  study  and  used  to  record  responses  from  interviewees. 

C.  SELECTING  FIELD  TESTERS  TO  INTERVIEW 

Application  of  the  Harvard  Business  School’s  Redesigning  Product/Service 
Development  program  field  tester  selection  criteria  was  relatively  straightforward  as  applied 
to  the  submarine  customer  product  development  interview  planning.  The  first  step  was  to 
define  users  versus  purchasers.  For  the  submarine  C4I  systems  of  interest,  the  users  are  the 
officers  and  enlisted  communications  technicians  assigned  to  operational  units  whereas  the 
purchasers  are  program  office  and  resource  sponsor  personnel.  As  outlined  in  the  L.L.  Bean 
method,  the  product  development  interviews  were  limited  to  the  user  community.  The 
second  step  in  selecting  submarine  field  testers  is  to  define  experienced  users.  In  this  case, 
the  level  of  experience  with  the  submarine  C4I  systems  of  interest  was  the  determining 
factor.  Experienced  users  were  defined  as  those  submarine  operators  who  had  recently 
returned  from  a  six- month  deployment  while  equipped  with  those  systems. 

Defining  demographic  categories  in  which  diverse  representation  among  experienced 
users  is  desired  constitutes  step  three  of  the  field  tester  selection  process.  For  submarine  C4I 
systems,  the  important  demographic  diversity  consideration  is  the  rank  of  selected  field 
testers.  It  is  desirable  to  include  experienced  users  from  all  ranks  that  operate  the  equipment 
or  depend  upon  its  operation  to  execute  the  submarine’s  mission.  This  results  in  selection  of 
a  broad  cross-section  of  ranks  onboard  from  the  junior  enlisted  communicators  (typically  a 
third  class  petty  officer/E-4)  to  the  Commanding  Officer  (a  Commander/0-5).  Other 
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possible  demographic  categories  (gender,  geographic  location,  etc.)  were  determined  to  be 
unimportant  for  this  study. 

The  final  step  in  determining  field  tester  selection  criteria  is  to  define  common 
characteristics  for  experienced,  demographically  diverse  users.  In  this  regard,  L.L.  Bean’s 
set  of  common  characteristics  for  field  testers  was  directly  applicable  to  the  desired 
characteristics  of  submarine  field  testers.  Specifically,  submarine  field  testers  were  pursued 
who  were  articulate,  analytical,  honest,  unbiased  and  timely  in  providing  feedback. 

In  addition  to  applying  the  Harvard  Business  School’s  step-by-step  method  to 
defining  submarine  field  tester  selection  criteria,  two  additional  considerations  specific  to  the 
submarine  community  were  used  to  help  select  and  recruit  field  testers  for  this  study.  These 
considerations  are  discussed  below. 

1.  Submarine  Deployment  Cycles 

The  typical  submarine  training  and  deployment  schedule  includes  eighteen  months  of 
in  port  and  underway  training  in  the  vicinity  of  the  ship’s  homeport  followed  by  a  six- month 
duration  overseas  deployment.  The  crew’s  operating  expertise  in  all  areas,  including 
communications,  can  be  expected  to  peak  during  the  overseas  deployment  when  the 
operational  tempo  normally  requires  the  submarine  to  be  underway  about  75-80  percent  of 
the  time.  Additionally,  the  most  demanding  and  intensive  communications  scenarios  occur 
during  the  overseas  deployment.  For  these  reasons,  the  ideal  submarine  field  testers  would 
have  recently  completed  an  overseas  deployment. 
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2.  Submarine  Community  Input 

In  order  to  ensure  that  selected  submarine  field  testers  would  be  the  most  proficient 
and  knowledgeable  communications  personnel  available,  the  submarine  force  type 
commanders  in  the  Atlantic  and  Pacific  fleets  were  consulted  and  asked  to  provide  their 
perspective  regarding  which  submarines  had  consistently  demonstrated  the  highest  level  of 
communications  expertise.  Since  the  type  commander  personnel  have  detailed  knowledge  of 
the  communications  proficiency  for  all  submarines  under  their  command,  this  input  was 
heavily  weighted  in  selecting  the  submarine  field  testers  for  this  study. 

Applying  the  tenets  of  the  Harvard  Business  School  process  and  including  inputs 
from  submarine  force  operational  commanders,  four  submarines  were  selected  for  this  study: 
USS  JEFFERSON  CITY  (SSN  759),  USS  ASHEVIFFE  (SSN  758),  USS  SANTA  FE  (SSN 
763)  and  USS  OFYMPIA  (SSN  718).  Twenty- two  submarine  field  testers  were  chosen  from 
the  crews  of  these  ships,  including  two  Commanding  Officers,  two  Operations  Officers,  four 
Communications  Officers,  four  Communications  Feading  Chief  Petty  Officers,  five 
Communications  Watch  Supervisors  and  eight  Communications  Watch  Standers. 

D.  CONDUCT  OF  INTERVIEWS 

Several  key  tenets  of  the  Harvard  Business  School’s  Redesigning  Product/Service 
Development  program  to  obtain  the  customer  voice  describe  attributes  that  are  specific  to  the 
conduct  of  the  interview  itself.  The  following  paragraphs  describe  how  the  tenets  of  “Go 
right  to  your  customers  turf’,  “Fisten  to  your  customers  face-to-face”  and  “Watch  your 
customers  actually  using  your  products  or  services”  were  applied  during  the  submarine 
communications  product  development  customer  interviews  (Garvin,  2000,  p.  11). 
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1. 


Interview  Locations 


For  the  submarine  eommunity,  the  eustomer’s  turf  is  easily  defined  as  the  submarine 
itself.  All  submarine  eustomer  produet  development  interviews  for  this  study  were 
eondueted  faee-to-faee  with  seleeted  field  testers  onboard  operational  fast  attaek  submarines. 
Interestingly  however,  the  different  demographie  groups  (offieer  and  enlisted  personnel) 
require  slightly  different  loeations  onboard  the  submarine  to  feel  more  at  ease  during  the 
interviews.  For  officers,  the  interview  location  chosen  is  the  submarine  wardroom  where  the 
officers  typically  meet  for  meals  and  training  evolutions  on  an  underway  submarine.  The 
wardroom  offers  a  relaxed  atmosphere  where  the  officers  can  comfortably  describe  their 
experiences  with  the  C4I  systems  of  interest  to  this  study.  For  enlisted  communications 
technicians,  the  radio  room  offers  the  best  area  to  interface  with  the  selected  field  testers  as  it 
makes  them  feel  at  ease  in  their  working  environment  and  affords  the  opportunity  for  them  to 
demonstrate  the  operation  of  the  products  during  the  conduct  of  the  interview  (another  key 
tenet  of  the  L.L.  Bean  method).  All  of  the  submarine  C4I  systems  of  interest  to  this  study  are 
physically  located  in  the  submarine’s  radio  room. 

2.  Interview  Schedule 

The  Harvard  Business  School’s  process  for  interviewing  field  testers  requires  that  the 
interviews  take  no  more  than  one  hour  of  the  field  tester’s  time.  This  helps  ensure  that  the 
field  testers  enjoy  the  process  and  don’t  feel  burdened  by  it.  The  L.L.  Bean  method  relies  on 
a  small  group  of  motivated  field  testers  to  be  used  again  and  again  to  help  design  new 
products  and  improve  existing  ones.  Field  testers  should  perceive  the  interviews  as  an 
opportunity  to  contribute  their  ideas  to  the  development  of  superior  products  rather  than  just 
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another  task  to  add  to  their  already  lengthy  daily  routine.  It  is  particularly  important  to 
follow  this  guideline  when  working  with  submarine  field  testers  since  they  are  extremely 
busy  with  the  day-to-day  operations  of  the  submarine.  All  of  the  interviews  planned  for  this 
study  were  scheduled  for  a  one -hour  duration  in  keeping  with  the  Redesigning 
Product/Service  Development  program  criteria  and  to  limit  the  impact  on  the  submarine’s  in 
port  schedule. 

E.  CHAPTER  SUMMARY 

This  chapter  has  described  three  main  areas  of  research  methodology  required  in  this 
thesis.  The  first  area  involved  planning  the  product  development  interviews  to  include 
determining  the  composition  of  the  interview  team  and  developing  the  questions  to  be  used 
during  the  interview.  The  second  area  consisted  of  selecting  field  testers  from  the  submarine 
community  to  interview.  Finally,  the  third  area  was  the  conduct  of  the  interviews 
themselves.  The  results  of  these  interviews  will  be  critical  in  determining  the  applicability  of 
the  Harvard  Business  School’s  Redesigning  Product/Service  Development  program  to 
military  C4I  systems  development  and  to  develop  useful  product  requirements  for  the 
submarine  C4I  systems  of  interest.  The  following  chapters  analyze  and  summarize  the 
results  of  these  submarine  customer  product  development  interviews. 


37 


THIS  PAGE  INTENTIONALLY  LEET  BLANK 


38 


IV.  DATA  ANALYSIS  AND  INTERVIEW  RESULTS 


A.  INTRODUCTION 

This  chapter  analyzes  interview  responses  from  submarine  field  testers.  Selected 
submarine  crew  members  answered  five  open-ended  questions  regarding  the  submarine  C4I 
systems  of  interest  and  provided  recommendations  for  their  improvement.  The  interview 
results  were  analyzed  in  accordance  with  the  Harvard  Business  School’s  Redesigning 
Product/Service  Development  method.  This  chapter  discusses  the  post-  interview  debriefings 
conducted  immediately  following  each  series  of  interviews  and  describes  how  submarine 
field  tester  interview  results  are  translated  into  summarized  quotations  and  descriptions. 
These  summarized  results  are  then  translated  into  product  requirements  and  design  ideas 
following  the  L.L.  Bean  model. 

B.  INTERVIEW  RESULTS 

The  data  for  this  thesis  are  the  submarine  field  tester  comments  recorded  during  one- 
on-one  interviews  with  them.  Twenty- two  carefully  selected  submarine  crew  members  were 
interviewed  using  the  L.L  Bean  product  development  interview  techniques  as  described 
above.  Interviewee  responses  were  recorded  exactly  without  interpretation,  filtering  or 
summary  as  recommended  in  the  Harvard  Business  School’s  Redesigning  Eroduct/Service 
Development  Toolkit  (Garvin,  2000).  Post- interview  debriefings  were  conducted  as  soon  as 
possible  following  each  series  of  interviews,  and  follow-up  meetings  were  held  to  summarize 
and  translate  submarine  field  tester  interview  results  into  product  requirements  and  design 
ideas. 
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1.  Post-Interview  Debriefings 

The  Harvard  Business  School  program  suggests  that  the  interviewer  and  recorder 
conduct  post- interview  debriefings  with  one  another  as  soon  as  possible  following  each 
interview.  This  allows  the  team  to  review  the  interview  notes  and  fill  in  any  gaps  while  the 
interview  is  still  fresh  on  their  minds.  The  objective  of  the  debriefings  was  to  capture  what 
the  submarine  field  testers  thought  and  felt  most  deeply  about  the  submarine  C4I  systems  of 
interest  by  selecting  the  most  evocative  quotations  and  vivid  descriptions  they  provided 
during  the  interview.  Examples  of  these  quotations  regarding  the  submarine  C4I  systems  of 
interest  include: 

“Too  many  things  that  we  are  getting  --  a  bunch  of  geek  toys  --  do  we  really 
need  them  all?” 

“The  installation  process  --  they  install  equipment,  they  give  no  tech  support, 
they  give  us  on- the- fly  training,  they  say  good  bye  and  have  fun.  Then,  the 
equipment  breaks  down  and  sits  idle  for  the  rest  of  the  deployment.” 

“Biggest  problem  is  lack  of  time  for  training  --  knowledge  not  used  is 
knowledge  not  gained.” 

“All  of  this  shit  is  hard  to  fix  -  none  of  it  is  easy!” 

Appendix  B  provides  complete  submarine  customer  product  development  interview 
debriefing  results  arranged  by  interview  question  and  submarine  C4I  system  of  interest. 

2.  Summarizing  field  tester  Quotations  and  Descriptions 

In  accordance  with  the  Harvard  Business  School  method,  submarine  field  tester 
interviews  are  summarized  and  translated  in  three  separate  phases.  The  first  phase  involves 
summarizing  and  pooling  field  tester  quotations  and  descriptions  into  general  categories.  As 
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an  initial  step,  quotations  and  descriptions  are  pooled  according  to  the  interview  question  and 
system  of  interest  to  which  they  apply.  These  pooled  quotations  and  descriptions  are  then 
prioritized  and  reduced  according  to  the  interview  team’s  understanding  of  field  tester 
priorities.  Any  questions  regarding  field  tester  priorities  are  resolved  by  referring  to  the 
interview  transcripts.  This  process  continues  until  the  quotations  and  descriptions  are 
reduced  to  a  manageable  number,  approximately  three  to  five  times  the  number  of  team 
members  as  defined  by  the  Redesigning  Product/Service  Development  Toolkit  (Garvin, 
2000).  Next,  prioritized  quotations  and  descriptions  are  rearranged  into  clusters  by  themes, 
ignoring  the  interview  questions.  Any  quotations/descriptions  not  grouped  through  this 
process  are  evaluated  separately  according  to  their  perceived  importance,  referring  to  the 
interview  transcript  as  required.  These  ungrouped  quotations/descriptions  are  then  either 
eliminated  or  if  deemed  important  enough,  they  each  become  a  separate  group  of  one. 
Finally,  each  cluster  of  submarine  field  tester  quotations  and  descriptions  are  summarized 
with  a  brief  phrase  or  statement. 

For  each  of  the  submarine  C4I  systems  of  interest,  application  of  this  process  led  to 
the  following  field  tester  quotation/description  summary  statements: 
a.  UHF  MDR  Capability 

1)  Only  One  User  Allowed  at  a  Time 

2)  Inadequate  Operator  Training 

3)  System  Too  Slow 

4)  Spare  Parts  Support  Inadequate 

5 )  Poor  Installation  Planning 

6)  No  Line  of  Sight  Internet  Protocol  Communications  Capability 
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b.  Mini-DAMA  UHF  Transceiver: 


1 )  Spare  Parts  Support  Inadequate 

2)  Inadequate  Operator  Training 

3)  Insufficiently  Strong  Connectors  Used 

4)  Proliferation  of  Laptop  Computers  in  Radio  Room  is  a  Problem 

5 )  Poor  Backwards  Compatibility  with  Earlier  Systems 
c.  Submarine  Tactical  Data  Links: 

1 )  Systems  Not  Operationally  Useful 

2)  Inadequate  Operator  Training/Technical  Support 

3 )  Inadequate  Stowage  Available  for  Hard  Copy  Technical  Manuals 
Recurrence  of  summary  statements  regarding  inadequate  spare  parts  support  and 

operator  training  over  multiple  systems  is  likely  indicative  of  a  systemic  problem  with 
submarine  C4I  system  development.  Appendix  C  provides  the  complete  listing  of 
summarized  grouped  quotations  and  descriptions  developed  from  the  submarine  product 
development  interviews  as  part  of  this  study. 

3.  Translating  Summarized  Quotations  and  Descriptions  into  Product 

Requirements 

The  second  phase  of  the  Harvard  Business  School’s  Redesigning  Product/Service 
Development  method  to  translate  field  tester  interview  results  into  design  ideas  involves 
developing  product  requirements  from  the  summarized  field  tester  quotations  and 
descriptions.  First,  each  quotation/description  summary  statement  developed  above  is 
translated  into  one  or  more  product  requirements.  These  product  requirements  are  then 
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prioritized  and  reduced  using  a  process  similar  to  that  used  to  prioritize  and  reduce  submarine 
field  tester  quotations  and  descriptions.  Questions  regarding  field  tester  priorities  are 
resolved  by  referring  to  interview  transcripts  or  quotation/description  sheets.  In  this  way, 
product  requirements  are  reduced  to  a  manageable  number  of  approximately  three  to  five 
times  the  number  of  team  members  (Garvin,  2000).  Prioritized  requirements  are  then 
grouped  into  clusters  by  themes  and  any  ungrouped  requirements  are  evaluated  for 
elimination  or  retention  based  on  their  importance,  referring  to  quotation/description  sheets 
as  necessary.  Requirements  are  reviewed  to  ensure  submarine  field  tester  priorities  are 
accurately  captured  and  no  redundancies  exist  until  only  essential  requirements  remain. 

For  each  of  the  submarine  C4I  systems  of  interest,  application  of  this  process  led  to 
the  following  product  requirements: 

a.  UHF  MDR  Capability 

1.  System  shall  allow  for  multiple  simultaneous  users. 

2.  Operator  training  shall  address  all  facets  of  system  operation. 

3.  Operator  training  shall  be  made  available  to  all  planned  operators. 

4.  System  shall  allow  for  higher  data  throughputs. 

5.  System  sparing  shall  provide  adequate  support  for  deployed  operations. 

6.  System  installations  shall  be  planned  to  minimize  submarine  schedule 
impacts. 

7.  System  shall  provide  a  Line  of  Sight  Internet  Protocol  communications 
capability. 

b.  Mini-DAMA  UHF  Transceiver 

1 )  System  sparing  shall  provide  adequate  support  for  deployed  operations. 
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2)  Operator  training  shall  address  all  facets  of  system  operation. 

3)  Operator  training  shall  be  made  available  to  all  planned  operators. 

4)  System  connectors  shall  withstand  normal  wear  and  tear  to  include  routine 
maintenance  requirements. 

5)  Operating  workstation  for  the  system  shall  be  environmentally  qualified  with 
respect  to  anticipated  shock  and  vibration  conditions. 

6)  Operating  workstations  in  the  Radio  Room  shall  be  consolidated  to  the 
maximum  extent  possible. 

7)  System  shall  be  interoperable  with  UHF  Transceivers  in  service  on  other 
Navy  units. 

c.  Submarine  Tactical  Data  Links 

1 )  Systems  fielded  shall  be  operationally  useful  for  submarines. 

2)  N on-op  e  rationally  useful  systems  shall  be  removed  from  submarine  Radio 
Rooms. 

3)  Operator  training  shall  address  all  facets  of  system  operation. 

4)  Operator  training  shall  be  made  available  to  all  planned  operators. 

5)  Adequate  stowage  shall  be  provided  for  all  required  hard  copy  technical 
manuals  and/or  technical  manuals  shall  be  provided  in  electronic  format. 

Requirements  for  improved  and  more  readily  available  training  flow  naturally  from 
weaknesses  identified  in  all  three  submarine  C4I  systems  of  interest.  Improved  logistics 
support  is  also  a  recurring  requirement  across  multiple  systems.  Appendix  D  presents  the 
complete  product  requirements  listing  developed  for  the  submarine  C4I  systems  of  interest  as 


part  of  this  study. 
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4.  Translating  Product  Requirements  into  Design  Ideas 

The  final  phase  of  summarizing  and  translating  field  tester  interviews  in  accordance 
with  the  L.L.  Bean  method  begins  with  brainstorming  to  produce  many  product  design  ideas 
for  each  requirement  developed  in  the  previous  phase.  To  encourage  innovation,  no  design 
idea  should  be  considered  too  “crazy”  for  consideration.  Generated  design  ideas  are  then 
pooled  and  reduced  in  a  process  similar  to  that  used  for  field  tester  quotation/descriptions  and 
product  requirements.  Design  ideas  are  then  grouped  into  clusters  by  themes,  and  any  ideas 
not  grouped  are  evaluated  for  elimination  based  on  their  ability  to  meet  the  product 
requirements.  The  resulting  design  idea  listing  is  compared  to  the  submarine  field  tester 
quotation/description  sheets  to  ensure  that  the  ideas  and  concerns  of  the  experienced, 
demographic  ally  diverse  product  users  are  driving  the  design  idea  development  (Garvin, 
2000). 

For  each  of  the  submarine  C4I  systems  of  interest,  application  of  this  process  led  to 
the  following  design  ideas: 

a.  UHF  MDR  Capability 

1 )  Multi-User  UHF  MDR  Network  -  software  and  hardware  upgrades  to  allow 
more  than  one  submarine  user  at  a  time. 

2)  Mobile  Training  Team  -  Team  of  system  experts  dispatched  to  submarines 
soon  after  upgrades  installed  to  provide  comprehensive  training  (classroom 
and  underway  operational  training). 

3)  Advanced  Digital  Waveform  -  increase  system  throughput. 

4)  Improved  logistics  planning  -  enhance  spares  support. 
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5)  Submarine  Installation  Planning  Team  -  gather  installation  lessons  learned 
and  improve  planning  process. 

6)  Battle  Force  E-mail  System  -  Add  this  Government  Off-The-Shelf  (GOES) 
system  to  submarine  Radio  Rooms  to  provide  a  Line  of  Sight  Internet  Protocol 
communications  capability. 

b.  Mini-DAMA  UHF  Transceiver 

1)  Mobile  Training  Team  -  Team  of  system  experts  dispatched  to  submarines 
soon  after  upgrades  installed  to  provide  comprehensive  training  ( classroom 
and  underway  operational  training). 

2)  Improved  logistics  planning  -  enhance  spares  support. 

3)  Stronger  system  connectors  -  upgraded  connectors  to  provide  added  strength. 

4)  Rugged,  consolidated  workstations  -  eliminate  problems  with  laptops  and 
reduce  number  of  workstations. 

5)  Standard  Operating  Procedure  improvements  -  describe  equipment  setups 
required  to  ensure  interoperability  with  other  in  service  UHF  transceivers. 

c.  Submarine  Tactical  Data  Links 

1 )  Common  Operating  Picture  ( COP)  -  new  submarine  tactical  data  transfer 
paradigm  required  as  current  legacy  systems  are  clearly  not  useful  in  today ’s 
submarine  operations. 

2)  Mobile  Training  Team  -  Team  of  system  experts  dispatched  to  submarines 
soon  after  upgrades  installed  to  provide  comprehensive  training  ( classroom 
and  underway  operational  training). 
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3)  Interactive  Electronic  Technical  Manuals  (lETMs)  -  convert  hard  copy 
technical  manuals  to  electronic  format. 

Mobile  Training  Team  visits  could  enhance  operator  training  for  all  submarine  C4I 
systems  of  interest.  Improved  logistics  planning  would  also  provide  significant  benefits 
across  multiple  systems.  Appendix  E  lists  all  submarine  C4I  system  of  interest  design  ideas 
developed  as  a  result  of  this  study. 

E.  CHAPTER  SUMMARY 

The  Harvard  Business  School’s  Redesigning  Product/Service  Development  process 
has  been  applied  to  develop  new  design  ideas  for  submarine  C4I  systems  of  interest.  This 
chapter  described  the  phases  involved  in  the  application  of  this  method  to  translate  submarine 
field  tester  interview  results  into  product  requirements  and  design  ideas.  These  design  ideas 
can  be  used  to  improve  the  C4I  systems  of  interest  and  enhance  the  development  of  new 
submarine  C4I  systems  with  similar  requirements.  The  following  chapters  provide  lessons 
learned  and  conclusions  regarding  the  application  of  the  Harvard  Business  School’s 
methodology  to  military  C4I  system  development  and  give  recommendations  for  future 
research. 
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V.  EVALUATION  OF  HARVARD  BUSINESS  SCHOOL  PROGRAM  APPLIED  TO 
MILITARY  C4I  SYSTEMS  DEVELOPMENT 

A.  INTRODUCTION 

The  preceding  chapters  have  detailed  an  application  of  the  Harvard  Business  School’s 
Redesigning  Product/Service  Development  program  to  obtain  customer  inputs  for  submarine 
C4I  systems  development.  This  chapter  will  identify  lessons  learned  through  applying  the 
L.L.  Bean  method  for  developing  new  product  ideas  to  submarine  systems  and  generalize 
these  lessons  learned  to  other  military  systems.  Benefits  and  drawbacks  of  this  method  for 
obtaining  operating  forces’  input  to  C4I  systems  development  will  be  identified  and  obstacles 
to  employing  these  methods  in  the  military  services  will  be  explored.  Conclusions  and 
recommendations  for  further  research  will  then  be  provided  in  the  final  chapter  of  this  paper. 

B.  BENEFITS  OF  L.L  BEAN  MODEL 

The  greatest  advantage  of  the  Harvard  Business  School’s  Redesigning 
Product/Service  Development  model  as  applied  to  military  C4I  systems  development  is  its 
focus  on  obtaining  product  design  ideas  from  the  true  experts  in  the  field,  those  that  actually 
use  the  systems  to  accomplish  their  assigned  missions.  By  providing  a  logical  and 
comprehensive  framework  for  capturing  the  experiences  (both  good  and  bad)  of  the  actual 
customers,  the  Harvard  Business  School  method  enables  system  developers  to  capture  the 
true  mission  need  that  the  system  should  fulfill  and  significantly  enhances  the  developers’ 
understanding  of  the  system’s  operational  requirements.  By  immersing  product  developers 
in  their  customers’  experiences  through  face-to-face  interviews  on  the  customer’s  turf  with 


49 


an  opportunity  to  watch  customers  actually  using  the  C4I  systems  of  interest,  the  L.L.  Bean 
method  enhances  the  intensity  of  the  developer  and  customer  interactions.  Gruner  and 
Homburg  (2000)  have  developed  a  theoretical  justification  for  the  positive  influence  of 
customer  interaction  in  the  product  development  process  using  resource  dependence  theory, 
and  they  empirically  supported  their  theory  that  the  intensity  of  customer  interaction  in  the 
product  development  process  positively  impacts  product  success  through  intensive  study  of 
the  German  machinery  industry.  Their  research  also  suggests  that  customer  interaction  is 
most  effective  in  developing  products  where  a  high  degree  of  innovation  is  required  and 
where  the  customers  have  a  high  level  of  expertise  with  respect  to  the  product.  Both  of  these 
criteria  apply  strongly  to  military  C4I  systems,  indicating  the  importance  of  involving 
operational  users  in  the  product  development  process  and  implying  that  the  customer  focus 
inherent  in  the  L.L.  Bean  method  should  lead  to  product  development  success. 

Another  important  benefit  of  the  Harvard  Business  School’s  Redesigning 
Product/Service  Development  program  is  their  methodical  approach  to  identifying  customers 
with  an  optimal  mix  of  expertise  and  experience  to  serve  as  field  testers  and  provide  the 
necessary  user  insights  into  the  development  process.  Gruner  and  Homburg’ s  (2000) 
research  also  supports  the  hypothesis  that  the  characteristics  of  the  involved  customers  have  a 
direct  impact  on  product  development  success.  By  providing  a  detailed  framework  for 
identifying  experienced,  demographically  diverse  users  to  participate  in  the  product 
development  process,  the  Harvard  Business  School  method  maximizes  the  potential  for  a 
successful  outcome.  Figure  3  graphically  depicts  the  conceptual  framework  of  Gruner  and 
Homburg’ s  findings  with  respect  to  customer  interaction  and  new  product  success.  The  L.L. 
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Bean  method  integrates  both  criteria  identified  by  Gruner  and  Homburg  as  important  to 


successful  product  development. 


Figure  3:  Conceptual  Framework  for  Customer  Interaction  and  New  Product  Success 

(From:  Gruner  and  Homburg,  2000) 


A  final  important  benefit  of  the  Harvard  Business  School  program  is  the  step-by-step 
methodology  provided  to  translate  customer  interview  results  into  essential  product 
requirements  and  develop  design  ideas  to  precisely  meet  those  identified  requirements.  Each 
step  of  this  process  emphasizes  comparison  of  the  result  with  customer  statements  of 
necessary  product  features  to  ensure  that  the  ideas  and  concerns  of  experienced, 
demographically  diverse  product  users  are  continuing  to  drive  the  development  process.  This 
step  by  step  process  to  capture  customer  mission  needs  and  product  requirements  lends  itself 
well  to  submarine  C4I  systems  development  and  helps  to  ensure  that  successful  outcomes 
will  be  repeatable  over  a  wide  variety  of  military  C4I  systems  applications. 
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C.  LIMITATIONS  OF  L.L.  BEAN  MODEL 

A  significant  limitation  of  the  Harvard  Business  School’s  Redesigning 
Product/Service  Development  program  is  that  it  requires  that  experienced  customers  exist 
from  which  to  obtain  essential  product  requirements  and  stimulate  design  ideas.  While  this  is 
possible  in  most  product  development  applications,  highly  innovative  product  development 
efforts  may  have  difficulty  identifying  field  testers  to  interview.  For  example,  the  developer 
of  the  first  U.S.  Navy  submarine,  John  P.  Holland,  would  have  encountered  significant 
difficulty  in  identifying  experienced  submarine  users  to  participate  in  his  product 
development  process.  Thus,  there  likely  exists  an  upper  limit  to  the  capacity  of  the  L.L.  Bean 
method’s  ability  to  provide  useful  input  to  highly  innovative  military  C4I  systems 
development  efforts. 

Another  limitation  of  this  method  is  that  it  addresses  customer  interaction  with 
product  developers  only  in  the  early  design/concept  idea  stage  of  product  development. 
Gruner  and  Homburg’s  (2000)  research  results  encourage  interaction  in  the  late  stages  of  the 
product  development  process  (i.e.,  prototyping  and  product  launch)  as  well  as  the  early 
stages.  This  suggests  that  other  methods  in  addition  to  the  Harvard  Business  School  program 
are  required  to  maximize  the  benefits  of  customer  interaction  during  the  product  development 
process. 

C.  OBSTACLES  TO  IMPLEMENTING  THE  L.L.  BEAN  MODEL 

The  most  significant  obstacle  to  effective  implementation  of  the  Harvard  Business 
School’s  Redesigning  Product/Service  Development  program  in  military  C4I  systems 
development  is  the  DoD  acquisition  model  itself.  The  L.L.  Bean  method  is  premised  upon 
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direct  interaction  between  experienced  users  and  system  developers.  Although  the  DoD 
acquisition  model  encourages  and  requires  acquisition  and  requirements  community 
personnel  to  interact  with  the  user  community  to  enhance  new  product  developments,  it 
makes  no  specific  provisions  for  the  actual  system  developers  (i.e.,  defense  contractors)  to 
interact  on  a  systematic  basis  with  operational  users,  particularly  in  the  early  stages  of 
concept  development.  This  places  the  burden  for  capturing  and  translating  operational  user 
needs  and  requirements  on  the  acquisition  community  and  risks  inaccurately  or  inadequately 
conveying  the  user’s  intent  to  the  actual  system  developers.  Fortunately,  program  managers 
have  significant  leeway  in  implementing  DoD  regulations,  and  many  have  already  begun  to 
realize  the  significant  advanta^s  that  can  accrue  from  promoting  direct  interaction  between 
operational  users  and  system  developers.  For  example,  the  Submarine  Communications 
Program  Office  is  effectively  applying  this  technique  to  enhance  the  development  of  the 
VIRGINIA  (SSN  774)  Exterior  Communications  System  by  periodically  sending  submarine 
communications  electronics  technicians  assigned  to  operational  fleet  units  to  the  system 
developer’s  integration  facility,  allowing  contractor  system  and  software  engineers  to  better 
understand  the  operator  perspective  and  integrate  it  successfully  into  the  final  product. 

Other  less  significant  obstacles  to  effectively  implementing  the  Harvard  Business 
School  method  in  military  C4I  systems  development  include  funding  constraints  (both  the 
funds  required  to  implement  the  method  as  well  as  funds  required  to  implement  design  ideas 
identified  through  the  process)  and  the  training  required  to  teach  the  method  to  appropriate 
program  office  and  system  developer  personnel.  Efficient  budgeting  and  planning  practices 
by  the  program  office  and  contractor  facility  are  required  to  overcome  these  lesser  obstacles. 
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E.  CHAPTER  SUMMARY 

Benefits,  limitations  and  obstacles  to  implementing  the  Harvard  Business  School’s 
Redesigning  Product/Service  Development  process  have  been  outlined  in  this  chapter.  The 
L.L.  Bean  method  effectively  applies  the  tenets  of  intensity  of  customer  interaction  and 
characteristics  of  the  involved  customers  researched  by  Gruner  and  Homburg.  However,  this 
method  focuses  solely  on  the  early  stages  of  the  product  development  process,  and  it  focuses 
on  direct  interaction  between  operational  users  and  system  developers  that  can  be  harder  to 
achieve  in  military  systems  development.  The  final  chapter  provides  conclusions  regarding 
the  application  of  the  Harvard  Business  School’s  methodology  to  military  C4I  system 
development  and  gives  recommendations  for  future  research. 
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VI.  CONCLUSION  AND  RECOMMENDATIONS 


A.  CONCLUSION 

This  research  applied  a  commercial  product  development  process  developed  by  the 
Harvard  Business  School  and  L.L.  Bean,  Incorporated  to  military  C4I  systems  development. 
The  redesigning  product/service  development  program  developed  by  Garvin  focuses  on 
interviews  of  carefully  selected,  experienced  product  users  in  order  to  better  understand  the 
voice  of  the  customer  and  derive  new  product  requirements  and  design  ideas.  Open-ended 
questions  are  developed  and  asked  in  the  users’  environment  to  encourage  them  to  tell  a  story 
that  captures  the  essence  of  their  product  needs.  Interview  results  are  analyzed  and 
congealed  into  product  requirements  and  design  ideas  through  step-by-step  review  and 
analysis.  This  thesis  used  this  method  to  capture  the  voice  of  the  submarine  customer  as 
applied  to  communications,  command  and  control  systems. 

Research  for  this  thesis  included  a  review  of  current  best  commercial  practices  in 
injecting  customer  input  into  the  new  product  development  process.  Previous  research  on 
applying  commercial  product  development  processes  to  military  systems  development  was 
also  reviewed.  In  order  to  gather  submariner  input,  twenty- two  officers  and  enlisted 
personnel  currently  assigned  to  operational  submarines  were  selected  as  C4I  system  field 
testers.  These  experienced  users  were  interviewed  onboard  their  ships  to  capture  their 
perspective  on  system  requirements  and  ideas  for  improvement. 

The  interview  results  were  translated  into  concrete  product  requirements  and  design 
ideas  using  the  step-by-step  method  recommended  by  the  Harvard  Business  School  program. 
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Design  ideas  developed  during  the  course  of  this  research  are  currently  being  implemented  to 
improve  the  effectiveness  and  user  acceptance  of  submarine  C4I  systems.  Initial  results  of 
this  implementation  are  promising.  The  Submarine  C4I  Mobile  Training  Team,  in  particular, 
has  demonstrated  significant  success  in  increasing  the  ability  of  the  submarine  users  to 
operate  and  maintain  new  C4I  system  enhancements  effectively.  The  training  team  benefited 
greatly  from  the  data  collected  as  part  of  this  thesis,  incorporating  the  customer  insights 
directly  into  their  curriculum.  Feedback  from  their  subsequent  training  visits  to  submarines 
has  been  overwhelmingly  positive.  Space  and  Naval  Warfare  System  Command  program 
offices  are  currently  implementing  other  design  ideas  developed  through  this  research,  and 
it’s  still  too  early  to  judge  their  effectiveness  in  actual  submarine  force  operations. 

In  addition  to  leveraging  experienced  submarine  user  operational  knowledge  to 
develop  concrete  design  ideas  for  improving  submarine  C4I  systems,  the  larger  benefit  of  this 
thesis  is  the  successful  application  of  a  commercially  developed  product  development 
process  in  military  C4I  systems  development.  The  same  Harvard  Business  School  program 
that  enhances  L.L.  Bean’s  ability  to  integrate  customer  input  into  the  development  of  new 
hunting  boots  has  demonstrated  utility  in  capturing  experienced  military  user’s  C4I  system 
product  requirements  and  design  ideas  as  well.  The  basic  tenet  of  the  redesigning 
product/service  development  program  holds  that  the  customer,  not  the  product  designer,  is 
the  true  system  expert.  So,  it  is  obviously  important  to  make  every  effort  to  accurately 
capture  the  voice  of  the  customer  early  and  often  in  the  new  product  development  process. 
The  DoD  acquisition  system  also  recognizes  the  importance  of  customer  (in  this  case,  the 
warfighter)  input  into  the  new  product  development  process.  The  Mission  Needs  Statement, 


developed  by  the  operating  forces,  marks  the  initiation  of  any  new  system  development. 

56 


However,  as  many  successful  commercial  companies  have  come  to  realize  and  the  DoD  is 
beginning  to  understand,  a  Mission  Needs  Statement  alone  is  an  inadequate  vehicle  for 
completely  capturing  the  voice  of  the  customer.  A  continuing  dialogue  between  the  user, 
requirements  and  acquisition  communities  is  required  to  ensure  that  system  developers  truly 
understand  the  customer’s  needs  and  priorities.  Further,  the  defense  contractors  who  actually 
develop  military  systems  need  to  be  included  in  this  dialogue.  This  will  become  increasingly 
important  as  we  strive  to  transform  the  military  and  our  acquisition  processes.  Admiral 
Dennis  Blair,  Commander- in- Chief  of  the  Pacific  Command,  also  espouses  this  view, 
labeling  it  “Acquisition  by  Adaptation”: 

The  paradigm  for  the  future  should  be  based  on  quicker  partnerships  ... 
putting  a  prototype  system  out  quickly,  then  adapting  and  improving  it  as  it’s 
fielded.  The  only  way  to  do  this  is  to  connect  the  engineers  directly  with  the 
fleet  and  field  units,  with  the  acquisition  community  as  enablers  for  that 
process,  not  controllers  of  it,  and  the  CINC’s  identifying  requirements,  setting 
priorities  and  providing  venue  for  systems  development  (Blair,  2001,  p.  4). 

The  Submarine  Communications  Program  Office  is  embracing  this  vision  by  pursuing 
innovative  ways  to  inject  the  voice  of  the  submarine  warfighter  into  the  development  of  the 
VIRGINIA  (SSN  774)  Exterior  Communications  System,  maximizing  the  direct  interaction 
of  submarine  communications  technicians  assigned  to  operational  fleet  units  and  system 
developers  employed  by  Lockheed  Martin  under  contract  to  deliver  the  system.  As  an 
outgrowth  of  the  concepts  developed  during  the  course  of  this  thesis,  a  comprehensive  Fleet 
Operator  System  Development  and  Integration  Assistance  Team  has  been  chartered  to 
directly  connect  experienced  submarine  communicators  with  the  engineers  developing  the 
next  generation  of  submarine  communications  systems.  Qualified  communications 
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watchstanders  and  maintenance  technicians  from  afloat  units  identified  by  the  Commander, 
Naval  Submarine  Forces  in  Norfolk,  Virginia  will  periodically  travel  to  the  Lockheed  Martin 
Exterior  Communication  Systems  integration  facility  in  Eagan,  Minnesota  to  participate 
directly  in  system  development  and  integration.  By  placing  aibmarine  communications 
system  experts  from  the  user  community  on  site  with  system  developers,  this  initiative  should 
help  ensure  that  the  voice  of  the  submarine  warfighter  is  well  represented  in  the  final  Exterior 
Communications  System  delivered  to  the  Navy  and  enhance  overall  submarine  customer 
satisfaction. 


B.  RECOMMENDATIONS  FOR  FUTURE  RESEARCH 

The  Harvard  Business  School’s  Redesigning  Product/Service  Development  program 
is  only  one  example  of  a  commercial  product  development  process  that  can  be  successfully 
adapted  to  identify  and  inject  the  voice  of  the  warfighter  into  military  C4I  systems 
development.  In  support  of  the  DoD’s  continuing  quest  for  acquisition  excellence,  this  thesis 
provides  five  areas  to  be  considered  for  future  research. 

1.  Investigate  Web- based  methods  for  capturing  the  voice  of  the  warfighter. 

Drazen  Prelac’s  Information  Pump  (2001)  can  provide  the  voice  of  the  customer 
via  the  Web  and  enable  customers  to  compare  and  contrast  desirable  product 
features  via  web-based  interaction  with  one  another.  This  method  also  has  the 
distinct  advantage  of  incorporating  a  scoring  method  for  qualitative  and 
quantitative  feedback.  This  avenue  of  research  is  worth  pursuing  because  it  may 
capture  the  voice  of  the  warfighter  more  efficiently  and  with  less  cost  as 
compared  to  the  Harvard  Business  School’s  Redesigning  Product/Service 
Development  method. 

2.  Explore  new  methods  for  increasing  direct  interface  between  military  system 
users  (warfighters)  and  product  developers  (defense  contractor  engineers). 

Benefits  from  this  research  would  be  two-fold.  Product  developers  would  gain  a 
better  understanding  of  user  needs  and  requirements,  while  warfighters  would 
simultaneously  gain  a  better  appreciation  for  military  system  capabilities  and 
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limitations.  These  methods  should  therefore  lead  to  better  military  products  that 
are  more  effectively  employed  in  the  field. 

3.  Conduct  a  quantitative  analysis  of  the  tangible  benefits  of  injecting  the  voice 
of  the  warfighter  directly  into  the  new  product  development  process.  This 
study  would  entail  a  comparison  of  military  product  developments  that  used  a 
high  degree  of  experienced  user  input  during  the  development  process  with  those 
that  involved  warfighters  to  a  lesser  degree.  Quantitative  metrics  involving 
specific  performance  achievements  and  cost  reductions  would  be  developed  and 
analyzed. 

4.  Create  a  database  of  commercial  best  practices  that  have  been  proven  to 
enhance  military  systems  product  development.  The  product  of  this  research 
would  become  a  toolkit  allowing  government  program  managers  to  select 
commercial  techniques  that  have  demonstrated  success  in  military  applications. 

5.  Apply  the  Harvard  Business  School’s  Redesigning  Product/Service 
Development  program  methods  to  the  development  of  an  Operational 
Requirements  Document.  This  research  would  apply  the  same  method 
described  in  this  thesis  to  a  new  start  military  program  at  the  outset  to  determine  if 
the  same  type  of  one-on-one  interview  process  with  experienced  users  can  be 
useful  to  formulate  an  Operational  Requirements  Document  for  a  previously  non¬ 
existing  capability. 

Military  systems’  development  can  be  greatly  enhanced  by  more  effectively 
communicating  warfighter  requirements  to  the  engineers  tasked  with  designing  and  building 
them.  Commercial  methods  for  capturing  the  voice  of  the  customer  and  ingraining  it  into  the 
product  development  process  can  prove  helpful  in  military  applications.  Government 
program  managers  should  strive  to  increase  the  exchange  of  ideas  between  their  warfighter 
customers  and  the  engineers  actually  developing  the  systems  under  their  cognizance. 
Commercial  best  practices  such  as  the  Harvard  Business  School’s  Redesigning 
Product/Service  Development  program  can  help  them  do  it. 
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APPENDIX  A.  PRODUCT  DEVELOPMENT  INTERVIEW  DATA  FORMS 


a.  Describe  the  most 
demanding  communications 
period  that  occurred  during 
the  deployment  and  how  well 
or  how  poorly  the  new  C4I 
system  installations  supported 
the  ship  during  that  period? 

b.  Describe  the  most  difficult 
maintenance  evolution 
involving  the  new  C4I 
systems  that  occurred  during 
the  deployment  and  how  well 
or  how  poorly  the  technical 
documentation  supported  your 
maintenance  efforts. 

c.  Describe  the  training 
provided  during  the 
installation  of  the  new  C4I 
systems  and  how  well  or  how 
poorly  that  training  prepared 
you  for  operating  and 
maintaining  the  equipment 
during  your  deployment. 

d.  Describe  the  impact  of  the 
new  C4I  system  installation 
on  the  ship’s  routine/schedule. 

e.  Are  there  any  other  issues 
or  problems  with  the  new  C4I 
systems?  Were  any  Casualty 
Reports  (CASREPS) 
involving  the  new  C4I 
systems  required  during  the 
deployment? 
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APPENDIX  B.  PRODUCT  DEVELOPMENT  INTERVIEW  RESULTS 


a.  Describe  the  most  demanding  communications  period  that  occurred  during 
the  deployment  and  how  well  or  how  poorly  the  new  C4I  system  installations 
supported  the  ship  during  that  period? _ 

UHF  MDR  Capability: 

SIPRNET  to  CTF  12  ASW  WECAN  limited  by  satellite  time- sharing  with  other 
submarine,  time  broken  by  odd  hour  and  even  hour.  WECAN  used  by  a  lot  of 
players;  function  very  good  -  biggest  drawback  was  we  had  to  share. 

Continually  questioned  whether  or  not  the  data  got  off  the  ship;  an  example  was  when 
we  had  8  email  messages  received  we  had  no  indication  if  the  messages  were  sent 
really  transmitted. 

Server  at  SUBPAC  was  a  challenge;  MIESTAR  1  &  2  great  reach  back  to  SUBPAC; 
server  there  can  only  support  1  user  at  a  time;  degraded  capability  of  the  technology 
(UHF  MDR). 

Major  challenge  was  that  we  had  no  one  trained  on  the  exchange  server  -  only  one 
sailor  had  the  training,  and  he  left  the  ship  due  to  a  family  emergency. 

Time  sharing  a  challenge  -  cannot  always  connect  to  shore  server  when  desired. 

UHF  MDR  Capability  with  the  CHAT  software  is  a  neat  system. 

Convenient  way  to  make  required  reports. 

Biggest  limitation  is  that  only  one  submarine  can  use  it  at  a  time,  during  one  period 
the  SANTA  EE  used  it  for  8- 10  hours;  it  appears  it  is  first  come,  first  serve. 

So  slow  -  requires  too  much  periscope  depth  time  -extra  time  at  comms  depth  means 
mast  exposure  time  is  increased. 

Sent  overlays  this  way  during  COMPTUEX  --  had  to  share  time  with  other 
submarines.  Great  way  to  send  plans  and  overlays. 

Timeframe  for  the  use  of  UHF  MDR  Capability  controlled  by  SUBPAC 
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a.  Describe  the  most  demanding  communications  period  that  occurred  during 
the  deployment  and  how  well  or  how  poorly  the  new  C4I  system  installations 

supported  the  ship  during  that  period? _ 

Mini-DAMA  UHF  Transceiver: 

Eased  operability  of  Radio  Room  equipment  -  less  physical  twisting  and  turning. 

The  biggest  problem  was  the  interface  with  the  STENNIS  whose  crew  was  relatively 
new  and  had  not  operated  with  a  submarine  outfitted  with  Mini-Dama.  They  were 
not  familiar  with  the  dual  WSC  3  challenge.  In  Singapore  finally  figured  out  the 
difficulty  and  then  we  had  the  STENNIS  make  the  proper  adjustments.  Recommend 
that  the  Sub  Liaison  Officer  be  given  good  briefing  on  Mini-DAMAAVSC-3 
problems;  the  people  on  the  CV  need  education  of  the  capability  of  the  equipment  on 
the  submarine;  need  configuration  lineups  shared  with  the  BG. 

Mini-DAMA  was  slapped  on  in  a  day,  had  minima!  training  during  the  installation. 

Mini-DAMA  spare  cards  among  all  the  submarines  is  inconsistent  -  no  standard  spare 
equipment  given  with  the  installation;  no  standard  ready  spares;  lost  a  power  supply 
and  did  not  have  one  to  replace  it. 

Mini-DAMA  -  great  system! 

I  wish  I  had  DAMA  in  an  all  in  1  package  (referring  to  satellite  channels  assigned). 
Mini  DAMA  Line  of  Sight  communications  are  limited  in  range  for  UHE  Link  11. 
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a.  Describe  the  most  demanding  communications  period  that  occurred  during 
the  deployment  and  how  well  or  how  poorly  the  new  C4I  system  installations 
supported  the  ship  during  that  period? _ 

Submarine  Tactical  Data  Links: 

Did  not  utilize  S-TADIL- J  very  well  at  all  -  never  used  it. 

S-TADIL- J  -  Computer  glitch  caused  BG  Computer  to  go  down.  Training  &  Tech 
support  not  good  -  immature  system.  The  idea  that  this  new  technology  needs  to  be 
put  in  the  Fleet  quickly  with  no  manuals  or  tech  support  or  training  is  flawed. 

Had  on-the-job  training  for  2  or  3  days  during  the  Pre-Overseas  Movement  cycle  - 
this  was  5  months  prior  to  deployment  so  everything  the  sailors  learned  was  lost.  We 
had  very  little  documentation  and  the  installation  was  the  week  before  the 
deployment. 

S-TADIL-J  -  Installed  at  the  last  minute;  tested  the  system  during  installation  with 
the  shore  rather  than  out  at  sea  with  another  submarine  or  even  with  CV.  This  is  did 
not  help  us  at  all.  Spent  countless  days,  man-hours  and  contributed  to  intense 
frustration  trying  to  get  the  problem  resolved  with  NCTAMS  (sub  rate  is  19.4)  as  it 
turned  out  all  it  took  was  turning  a  switch  at  the  shore  site.  No  SOP  written  for  the 
shore  or  CV  to  work  with  the  sub  with  this  equipment.  Worthless  system. 

LINK  16  -  we  never  use  it.  It  stays  off,  it  is  like  a  big  heater;  did  not  use  the  entire 
deployment;  BG  needs  it  but  never  used  (with  us)  it  either,  did  not  get  any  training  on 
it.  Only  good  thing  about  it,  it  gave  the  boat  an  additional  handset  in  radio;  it  also 
grounded  out  the  BRA  34  antenna. 

LINK  1 1  -  Never  used  it  either. 

Subtract  some  equipment  from  the  radio  room  that  we  do  not  use. 

S-TADIL-J  -  the  hard  drives  never  worked  so  we  never  operationally  used  it  on  the 
deployment. 
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b.  Describe  the  most  difficult  maintenance  evolution  involving  the  new  C4I 
systems  that  occurred  during  the  deployment  and  how  well  or  how  poorly  the 
technical  documentation  supported  your  maintenance  efforts. _ 

UHF  MDR  Capability: 

Temporary  Alterations  -  no  support  for  them  -  when  you  lose  the  equipment  you 
lose  it  for  good  and  there  is  nothing  in  place  to  support  the  equipment. 

Laptops  -  If  you  lose  it  UHF  MDR  Capability  is  gone  -  need  bracket  to  hold  it. 
Cable  plug  in  (RJ45)  have  to  plug  into  the  computer  is  prone  to  breakage  and  is 
not  supported  by  onboard  spares. 

Need  Intra-BG  IP  connectivity  capability!  -  Look  at  LOS  IP  Capability  in  addition 
to  UHF  MDR  Capability 


Mini-DAMA  UHF  Transceiver: 

No  problems;  Tech  manual  tells  what  to  do. 

Hard  to  get  parts  -  lots  of  cannibalizations  from  ship-to-ship  required. 

Submarine  Tactical  Data  Links: 

No  space  to  store  manuals  -  need  to  have  technical  manuals  published  on  CD- 
ROM. 

All  of  this  shit  is  hard  to  fix  -  none  of  it  is  easy! 

All  equipment  is  too  hard  to  fix;  we  have  two  new  First  Class,  the  junior  sailors 
have  no  idea  about  the  technical  information;  do  not  have  time  to  send  all  sailors 
to  school  with  all  the  installations  in  the  schedule. 
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c.  Describe  the  training  provided  during  the  installation  of  the  new  C4I 
systems  and  how  well  or  how  poorly  that  training  prepared  you  for  operating 
and  maintaining  the  equipment  during  your  deployment. _ 

UHF  MDR  Capability: 

Insufficient  training  -  only  one  person  was  taught,  and  he  has  transferred  from 
ship. 

Training  is  effective  only  if  it  is  on-the-air  training,  especially  for  UHF  MDR. 

The  training  classes  that  we  did  have  provided  good  baseline  understanding  for 
networking  components. 

Underway  training  better  than  in  port  training,  but  operational  issues  can  impact 
training. 

Good  training,  but  a  little  too  in-depth. 

The  information  was  presented  well  since  those  that  did  attend  were  able  provide 
the  information  to  others. 

Training  is  beneficial  to  the  division  but  when  are  going  to  do  it? 

Biggest  problem  is  lack  of  time  for  training. 

Mini-DAMA  UHF  Transceiver: 

Need  practical  training  -  especially  on-the-air  formal  training. 

The  operators  need  to  see  how  it  really  works;  if  we  are  not  able  to  use  with  other 
ships,  we  can’t  see  if  our  transmissions  are  correct  or  if  the  equipment  is  operating 
the  right  way. 

Hands  on  training  is  good,  actually  operating  the  equipment  is  better. 

Hands-on  on-the-air  operation  necessary  to  really  learn. 

Good  operator  training  conducted  by  SSC-Charleston  (“I  would  not  have  been 
able  to  operate  and  maintain  Mini-DAMA  if  I  hadn’t  gone  to  Charleston”) 

Big  difference  between  great  training  next  to  pier  and  underway  -  operating  with 
the  equipment  is  the  absolute  best  training  possible. 
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c.  Describe  the  training  provided  during  the  installation  of  the  new  C4I 
systems  and  how  well  or  how  poorly  that  training  prepared  you  for  operating 
and  maintaining  the  equipment  during  your  deployment. _ 


Submarine  Tactical  Data  Links: 

S-TADIL- J  -  No  training  provided. 

Need  another  way  to  check  what  we  are  doing  (e.g.  after  a  transmission  contact 
the  other  unit  and  say  “these  are  my  settings  what  are  yours?).  Really  need  other 
ships’  support  to  do  effective  training. 

S-TADIL-J  -  Had  Saturday  morning  training;  no  operator  training;  the  training 
only  dealt  with  how  to  load  crypto. 

S-TADIL-J  -  2  hours  training,  I  think,  right  after  it  was  installed,  it  has  not  been 
turned  since  then  -  why  do  we  have  it  in  our  radio  room? 

Knowledge  not  used  is  knowledge  not  gained. 
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d.  Describe  the  impact  of  the  new  C4I  system  installation  on  the  ship’s 
routine/schedule. _ 

UHF  MDR  Capability: 

Need  to  front-  load  installations  during  upkeeps  to  the  maximum  extent  possible. 
Mini-DAMA  UHF  Transceiver: 

Entire  installation  was  challenging. 

Never  open  front  panel  door  -  no  reason  to  open  door.  Connectors  inside  are  very 
fragile  and  if  you  open  door,  there  is  a  good  chance  that  you  will  mess  them  up. 

Installation  occurred  during  SRA,  shipyard  routine  worked  well  with  the 
installation,  it  was  a  good  time  to  do  it. 

Submarine  Tactical  Data  Links: 

No  installation  issues. 
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e.  Are  there  any  other  issues  or  problems  with  the  new  C4I  systems?  were 
any  Casualty  Reports  (CASREPS)  involving  the  new  C4I  systems  required 
during  the  deployment? _ 

UHF  MDR  Capahility: 

UHF  MDR  capability  was  completely  successful. 

Data  Replication  took  too  long;  had  to  wait  until  replication  is  done  before 
bringing  up  UHF  MDR  Chat,  once  you  start  replication  you  can’t  do  anything  else 
-  replicate  takes  about  hour  to  an  hour  and  a  half. 

Too  many  things  that  we  are  getting  --a  bunch  of  geek  toys  --  do  we  really  need 
them  all? 

Mini-DAMA  UHF  Transceiver: 

Mini- DAM  A:  interoperability  challenge  is  the  transfer  of  information  between  the 
WSC3  Y2fV3  and  Mini-DAMA;  Have  to  have  2  WSC3  on  DAMA;  need  to  find 
out  what  works  (equipment)  and  once  it  works  do  not  swap  it  out. 

Laptop  Computer  is  the  weakest  link  for  the  Mini-DAMA  system. 

Most  of  the  problems  experienced  had  to  do  with  the  CVN  ,  more  training  and 
procedures  need  to  be  given  to  the  CVN  . 

Mini-DAMA  has  a  faulty  back  plane  issue. 

Need  a  back-up  laptop  for  Mini-DAMA  -  should  be  standard  parts  support. 
Computers  should  be  considered  consumables  like  repair  parts.  If  Mini-DAMA 
computer  goes  out,  you  have  no  spares.  If  it  goes  down,  you  lose  capability  — 
each  boat  should  a  spare. 

Need  to  consolidate  workstations  and  eliminate  proliferation  of  laptop  computers 
in  the  Radio  Room  (Note  -  Ship  had  installed  a  flat  screen  monitor  with  a 
computer  workstation  (vice  laptop)  as  the  Mini-DAMA  controller  (replaced  a 
broken  laptop)  for  better  presentation  and  a  central  location  of  two  displays  (Mini- 
DAMA  and  BBS). 
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e.  Are  there  any  other  issues  or  problems  with  the  new  C4I  systems?  were 
any  Casualty  Reports  (CASREPS)  involving  the  new  C4I  systems  required 

during  the  deployment? _ 

Submarine  Tactical  Data  Links: 

S-TADIL- J  -  Something  happened  to  the  hardware  outside  of  the  radio  room;  loss 
of  some  executable  files,  no  circuit  for  each,  never  used  the  system  -  tried  to  use, 
but  never  could  get  it  to  come  up. 

LINK  1 1  -  Not  used  in  Arabian  Gulf;  Receive  only  when  used  -  rarely 
operational. 

NO  STOWAGE  -  We  have  no  place  to  store  anything  in  the  radio  room.  Put  the 
tech  manuals  on  CD/electronic  format  that  will  help  alleviate  some  of  the 
problems. 

Stowage  is  a  major  issue  that  no  one  is  addressing. 

S-TADIL- J:  Installed,  but  not  tested. 

The  process  --  they  install  equipment,  they  give  no  tech  support,  they  give  us  on 
fly  training,  they  say  good  bye  and  have  fun,  the  equipment  breaks  down  and  it 
sits  idle. 
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APPENDIX  C.  SUMMARIZED  FIELD  TESTER  QUOTATIONS  AND  DESCRIPTIONS 


UHF  MDR  Capability; _ 

Summary  Statement  -  Only  One  User  Allowed  at  a  Time 

Time  sharing  a  challenge  -  cannot  always  connect  to  shore  server  when  desired. 

Biggest  limitation  is  that  only  one  submarine  can  use  it  at  a  time,  during  one  period 
the  SANTA  FE  used  it  for  8- 10  hours;  it  appears  it  is  first  come,  first  serve. 

SIPRNET  to  CTE  12  ASW  WECAN  limited  by  satellite  time- sharing  with  other 
submarine,  time  broken  by  odd  hour  and  even  hour.  WECAN  used  by  a  lot  of 
players;  function  very  good  -  biggest  drawback  was  we  had  to  share. 

Timeframe  for  the  use  of  UHE  MDR  Capability  controlled  by  SUBPAC 

Server  at  SUBPAC  was  a  challenge;  MIESTAR  1  &  2  great  reach  back  to  SUBPAC; 
server  there  can  only  support  1  user  at  a  time;  degraded  capability  of  the  technology 
(UHE  MDR). 

Sent  overlays  this  way  during  COMPTUEX  --  had  to  share  time  with  other 
submarines.  Great  way  to  send  plans  and  overlays. 

Summary  Statement  -  Inadequate  Operator  Training 

Major  challenge  was  that  we  had  no  one  trained  on  the  exchange  server  -  only  one 
sailor  had  the  training,  and  he  left  the  ship  due  to  a  family  emergency. 

Insufficient  training  -  only  one  person  was  taught,  and  he  has  transferred  from  ship. 

Training  is  effective  only  if  it  is  on-the-air  training,  especially  for  UHE  MDR. 

Underway  training  better  than  in  port  training,  but  operational  issues  can  impact 
training. 

Good  training,  but  a  little  too  in-depth. 

The  information  was  presented  well  since  those  that  did  attend  were  able  provide  the 
information  to  others. 

Training  is  beneficial  to  the  division  but  when  are  going  to  do  it? 

Biggest  problem  is  lack  of  time  for  training. 
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UHF  MDR  Capability  (continued): 


Summary  Statement  -  System  Too  Slow 

So  slow  -  requires  too  much  periscope  depth  time  -extra  time  at  comms  depth  means 
mast  exposure  time  is  increased. 

Data  Replication  took  too  long;  had  to  wait  until  replication  is  done  before  bringing 
up  UHF  MDR  Chat,  once  you  start  replication  you  can’t  do  anything  else  -  replicate 
takes  about  hour  to  an  hour  and  a  half. 


Summary  Statement  -  Spare  Parts  Support  Inadequate 

Temporary  Alterations  -  no  support  for  them  -  when  you  lose  the  equipment  you  lose 
it  for  good  and  there  is  nothing  in  place  to  support  the  equipment. 

Laptops  -  If  you  lose  it  UHF  MDR  Capability  is  gone  -  need  bracket  to  hold  it. 

Cable  plug  in  (RJ45)  have  to  plug  into  the  computer  is  prone  to  breakage  and  is  not 
supported  by  onboard  spares. 


Summary  Statement  -  Poor  Installation  Planning 

Too  many  things  that  we  are  getting  --a  bunch  of  geek  toys  --  do  we  really  need  them 
all? 

Need  to  front-  load  installations  during  upkeeps  to  the  maximum  extent  possible. 


Summary  Statement -No  Line  of  Sight  Internet  Protocol  Communications  Capability 

Need  Intra-BG  IP  connectivity  capability!  -  Look  at  LOS  IP  Capability  in  addition  to 
UHF  MDR  Capability 
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Mini-DAMA  UHF  Transceiver: 


Summary  Statement  -  Spare  Parts  Support  Inadequate 

Hard  to  get  parts  -  lots  of  cannibalizations  from  ship-to-ship  required. 

Mini-DAMA  spare  cards  among  all  the  submarines  is  inconsistent  -  no  standard  spare 
equipment  given  with  the  installation;  no  standard  ready  spares;  lost  a  power  supply 
and  did  not  have  one  to  replace  it. 

Need  a  back-up  laptop  for  Mini-DAMA  -  should  be  standard  parts  support. 

Computers  should  be  considered  consumables  like  repair  parts.  If  Mini-DAMA 
computer  goes  out,  you  have  no  spares.  If  it  goes  down,  you  lose  capability  —  each 
boat  should  a  spare. 

Summary  Statement  -  Inadequate  Operator  Training 

Mini-DAMA  was  slapped  on  in  a  day,  had  minimal  training  during  the  installation. 

Need  practical  training  -  especially  on-the-air  formal  training. 

The  operators  need  to  see  how  it  really  works;  if  we  are  not  able  to  use  with  other 
ships,  we  can’t  see  if  our  transmissions  are  correct  or  if  the  equipment  is  operating  the 
right  way. 

Hands  on  training  is  good,  actually  operating  the  equipment  is  better. 

Hands-on  on-the-air  operation  necessary  to  really  learn. 

Good  operator  training  conducted  by  SSC-Charleston  (“I  would  not  have  been  able  to 
operate  and  maintain  Mini-DAMA  if  I  hadn’t  gone  to  Charleston”) 

Big  difference  between  great  training  next  to  pier  and  underway  -  operating  with  the 
equipment  is  the  absolute  best  training  possible. 

Summary  Statement  -  Insujficiently  Strong  Connectors  Used 

Never  open  front  panel  door  -  no  reason  to  open  door.  Connectors  inside  are  very 
fragile  and  if  you  open  door,  there  is  a  good  chance  that  you  will  mess  them  up. 

Mini-DAMA  has  a  faulty  back  plane  issue. 
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Mini-DAMA  UHF  Transceiver  (continued): 


Summary  Statement  -  Proliferation  of  Laptop  Computers  in  Radio  Room  is  a 
Problem 

Laptop  Computer  is  the  weakest  link  for  the  Mini-DAMA  system. 

Need  to  consolidate  workstations  and  eliminate  proliferation  of  laptop  computers  in 
the  Radio  Room  (Note  -  Ship  had  installed  a  flat  screen  monitor  with  a  computer 
workstation  (vice  laptop)  as  the  Mini-DAMA  controller  (replaced  a  broken  laptop) 
for  better  presentation  and  a  central  location  of  two  displays  (Mini-DAMA  and  BBS). 


Summary  Statement  -  Poor  Backwards  Compatibility  with  Earlier  Systems 

The  biggest  problem  was  the  interface  with  the  STENNIS  whose  crew  was  relatively 
new  and  had  not  operated  with  a  submarine  outfitted  with  Mini-Dama.  They  were 
not  familiar  with  the  dual  WSC  3  challenge.  In  Singapore  finally  figured  out  the 
difficulty  and  then  we  had  the  STENNIS  make  the  proper  adjustments.  Recommend 
that  the  Sub  Eiaison  Officer  be  given  good  briefing  on  Mini-DAMAAVSC-3 
problems;  the  people  on  the  CV  need  education  of  the  capability  of  the  equipment  on 
the  submarine;  need  configuration  lineups  shared  with  the  BG. 

Mini- DAM  A:  interoperability  challenge  is  the  transfer  of  information  between  the 
WSC3  Y2fV3  and  Mini-DAMA;  Have  to  have  2  WSC3  on  DAMA;  need  to  find  out 
what  works  (equipment)  and  once  it  works  do  not  swap  it  out. 

Most  of  the  problems  experienced  had  to  do  with  the  CVN  ,  more  training  and 
procedures  need  to  be  given  to  the  CVN  . 
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Submarine  Tactical  Data  Links: 


Summary  Statement  -  Systems  Not  Operationally  Useful 
Did  not  utilize  S-TADIL- J  very  well  at  all  -  never  used  it. 

LINK  16  -  we  never  use  it.  It  stays  off,  it  is  like  a  big  heater;  did  not  use  the  entire 
deployment;  BG  needs  it  but  never  used  (with  us)  it  either,  did  not  get  any  training  on 
it.  Only  good  thing  about  it,  it  gave  the  boat  an  additional  handset  in  radio;  it  also 
grounded  out  the  BRA  34  antenna. 

LINK  1 1  -  Never  used  it  either. 

Subtract  some  equipment  from  the  radio  room  that  we  do  not  use. 

S-TADIL- J  -  the  hard  drives  never  worked  so  we  never  operationally  used  it  on  the 
deployment. 

LINK  1 1  -  Not  used  in  Arabian  Gulf;  Receive  only  when  used  -  rarely  operational. 

Summary  Statement  -  Inadequate  Operator  Training/Technical  Support 

S-TADIL- J  -  Computer  glitch  caused  BG  Computer  to  go  down.  Training  &  Tech 
support  not  good  -  immature  system.  The  idea  that  this  new  technology  needs  to  be 
put  in  the  Fleet  quickly  with  no  manuals  or  tech  support  or  training  is  flawed. 

Had  on-the-job  training  for  2  or  3  days  during  the  Pre-Overseas  Movement  cycle  - 
this  was  5  months  prior  to  deployment  so  everything  the  sailors  learned  was  lost.  We 
had  very  little  documentation  and  the  installation  was  the  week  before  the 
deployment. 

S-TADIL-J  -  Installed  at  the  last  minute;  tested  the  system  during  installation  with 
the  shore  rather  than  out  at  sea  with  another  submarine  or  even  with  CV.  This  is  did 
not  help  us  at  all.  Spent  countless  days,  man-hours  and  contributed  to  intense 
frustration  trying  to  get  the  problem  resolved  with  NCTAMS  (sub  rate  is  19.4)  as  it 
turned  out  all  it  took  was  turning  a  switch  at  the  shore  site.  No  SOP  written  for  the 
shore  or  CV  to  work  with  the  sub  with  this  equipment.  Worthless  system. 

No  space  to  store  manuals  -  need  to  have  technical  manuals  published  on  CD-ROM. 

All  of  this  shit  is  hard  to  fix  -  none  of  it  is  easy! 
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Summary  Statement  -  Inadequate  Operator  Training/Technical  Support  (continued) 

All  equipment  is  too  hard  to  fix;  we  have  two  new  First  Class,  the  junior  sailors  have 
no  idea  about  the  technical  information;  do  not  have  time  to  send  all  sailors  to  school 
with  all  the  installations  in  the  schedule. 

S-TADIL- J  -  No  training  provided. 

Need  another  way  to  check  what  we  are  doing  (e.g.  after  a  transmission  contact  the 
other  unit  and  say  “these  are  my  settings  what  are  yours?).  Really  need  other  ships’ 
support  to  do  effective  training. 

S-TADIL- J  -  Had  Saturday  morning  training;  no  operator  training;  the  training  only 
dealt  with  how  to  load  crypto. 

S-TADIL- J  -  2  hours  training,  I  think,  right  after  it  was  installed,  it  has  not  been 
turned  since  then  -  why  do  we  have  it  in  our  radio  room? 

S-TADIL- J  -  Something  happened  to  the  hardware  outside  of  the  radio  room;  loss  of 
some  executable  files,  no  circuit  for  each,  never  used  the  system  -  tried  to  use,  but 
never  could  get  it  to  come  up. 

S-TADIL-J:  Installed,  but  not  tested. 

The  process  --  they  install  equipment,  they  give  no  tech  support,  they  give  us  on  fly 
training,  they  say  good  bye  and  have  fun,  the  equipment  breaks  down  and  it  sits  idle. 

Summary  Statement  -  Inadequate  Stowage  Available  for  Hard  Copy  Technical 
Manuals 

NO  STOWAGE  -  We  have  no  place  to  store  anything  in  the  radio  room.  Put  the  tech 
manuals  on  CD/electronic  format  that  will  help  alleviate  some  of  the  problems. 

Stowage  is  a  major  issue  that  no  one  is  addressing. 
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APPENDIX  D.  SUBMARINE  C4I  SYSTEM  PRODUCT  REQUIREMENTS 


UHF  MDR  Capability: 

1.  System  shall  allow  for  multiple  simultaneous  users. 

2.  Operator  training  shall  address  all  facets  of  system  operation. 

3.  Operator  training  shall  be  made  available  to  all  planned  operators. 

4.  System  shall  allow  for  higher  data  throughputs. 

5.  System  sparing  shall  provide  adequate  support  for  deployed  operations. 

6.  System  installations  shall  be  planned  to  minimize  submarine  schedule  impacts. 

7.  System  shall  provide  a  Line  of  Sight  Internet  Protocol  communications  capability. 


Mini-DAMA  UHF  Transceiver: 

1.  System  sparing  shall  provide  adequate  support  for  deployed  operations. 

2.  Operator  training  shall  address  all  facets  of  system  operation. 

3.  Operator  training  shall  be  made  available  to  all  planned  operators. 

4.  System  connectors  shall  withstand  normal  wear  and  tear  to  include  routine  maintenance 
requirements. 

5.  Operating  workstation  for  the  system  shall  be  environmentally  qualified  with  respect  to 
anticipated  shock  and  vibration  conditions. 

6.  Operating  workstations  in  the  Radio  Room  shall  be  consolidated  to  the  maximum  extent 
possible. 

7.  System  shall  be  interoperable  with  UHF  Transceivers  in  service  on  other  Navy  units. 


Submarine  Tactical  Data  Links: 

1.  Systems  fielded  shall  be  operationally  useful  for  submarines. 

2.  Non- operationally  useful  systems  shall  be  removed  from  submarine  Radio  Rooms. 

3.  Operator  training  shall  address  all  facets  of  system  operation. 

4.  Operator  training  shall  be  made  available  to  all  planned  operators. 

5.  Adequate  stowage  shall  be  provided  for  all  required  hard  copy  technical  manuals  and/or 
technical  manuals  shall  be  provided  in  electronic  format. 
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APPENDIX  E.  SUMARINE  C4I  SYSTEM  DESIGN  IDEAS 


UHF  MDR  Capability: 

1.  Multi-User  UHF  MDR  Network  -  software  and  hardware  upgrades  to  allow  more  than 
one  submarine  user  at  a  time. 

2.  Mobile  Training  Team  -  Team  of  system  experts  dispatched  to  submarines  soon  after 
upgrades  installed  to  provide  comprehensive  training  (classroom  and  underway 
operational  training). 

3.  Advanced  Digital  Waveform  -  increase  system  throughput. 

4.  Improved  logistics  planning  -  enhance  spares  support. 

5.  Submarine  Installation  Planning  Team  -  gather  installation  lessons  learned  and  improve 
planning  process. 

6.  Battle  Force  E-mail  System  -  Add  this  Government  Off-The-Shelf  (GOTS)  system  to 
submarine  Radio  Rooms  to  provide  a  Line  of  Sight  Internet  Protocol  communications 
capability. 

Mini-DAMA  UHF  Transceiver: 

1 .  Mobile  Training  Team  -  Team  of  system  experts  dispatched  to  submarines  soon  after 
upgrades  installed  to  provide  comprehensive  training  (classroom  and  underway 
operational  training). 

2.  Improved  logistics  planning  -  enhance  spares  support. 

3.  Stronger  system  connectors  -  upgraded  connectors  to  provide  added  strength. 

4.  Rugged,  consolidated  workstations  -  eliminate  problems  with  laptops  and  reduce  number 
of  workstations. 

5.  Standard  Operating  Procedure  improvements  -  describe  equipment  setups  required  to 
ensure  interoperability  with  other  in  service  UHF  transceivers 


Submarine  Tactical  Data  Links: 

1 .  Common  Operating  Picture  (COP)  -  new  submarine  tactical  data  transfer  paradigm 
required  as  current  legacy  systems  are  clearly  not  useful  in  today’s  submarine  operations 

2.  Mobile  Training  Team  -  Team  of  system  experts  dispatched  to  submarines  soon  after 
upgrades  installed  to  provide  comprehensive  training  (classroom  and  underway 
operational  training). 

3.  Interactive  Electronic  Technical  Manuals  (lETMs)  -  convert  hard  copy  technical  manuals 
to  electronic  format 
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